-- Logs begin at Wed 2022-08-24 02:44:53 UTC, end at Wed 2022-08-24 03:55:12 UTC. -- Aug 24 03:16:09.719917 dv-cinder-854261-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 03:16:09.730592 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 03:16:09 2022] *** Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: nodename: dv-cinder-854261-1-iscsi Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: machine: x86_64 Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: clock source: unix Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: pcre jit disabled Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: detected number of CPU cores: 4 Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: current working directory: / Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: detected binary path: /usr/bin/uwsgi-core Aug 24 03:16:09.732603 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: your processes number limit is 47680 Aug 24 03:16:09.733232 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: your memory page size is 4096 bytes Aug 24 03:16:09.733232 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: detected max file descriptor number: 2048 Aug 24 03:16:09.733232 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: lock engine: pthread robust mutexes Aug 24 03:16:09.733232 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: thunder lock: enabled Aug 24 03:16:09.733232 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 03:16:09.733232 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 03:16:09.773256 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: Python main interpreter initialized at 0x562bf572e9c0 Aug 24 03:16:09.773256 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: python threads support enabled Aug 24 03:16:09.773256 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: your server socket listen backlog is limited to 100 connections Aug 24 03:16:09.773256 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: your mercy for graceful operations on workers is 90 seconds Aug 24 03:16:09.773581 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 03:16:09.773680 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: *** Operational MODE: preforking *** Aug 24 03:16:09.773950 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: *** uWSGI is running in multiple interpreter mode *** Aug 24 03:16:09.773950 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: spawned uWSGI master process (pid: 81474) Aug 24 03:16:09.774145 dv-cinder-854261-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 03:16:09.775598 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: spawned uWSGI worker 1 (pid: 81485, cores: 1) Aug 24 03:16:09.775598 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: spawned uWSGI worker 2 (pid: 81486, cores: 1) Aug 24 03:16:09.775598 dv-cinder-854261-1-iscsi devstack@keystone.service[81474]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 03:16:14.110204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:16:14.110204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: from cryptography.utils import int_from_bytes Aug 24 03:16:14.110995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:16:14.110995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: from cryptography.utils import int_from_bytes Aug 24 03:16:14.111746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:16:14.111746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: from cryptography.utils import int_from_bytes Aug 24 03:16:14.112192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:16:14.112192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: from cryptography.utils import int_from_bytes Aug 24 03:16:14.163012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 03:16:14.163182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 03:16:14.163303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] command line args: [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 03:16:14.163571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 03:16:14.163747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ================================================================================ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 03:16:14.163999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] admin_token = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.164153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 03:16:14.164233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config_dir = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.164349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 03:16:14.164435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.164541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] command line args: [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 03:16:14.164625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config_source = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.164746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] debug = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.164860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 03:16:14.164970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ================================================================================ {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 03:16:14.165231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] admin_token = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] config_dir = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] config_source = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.165955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] debug = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_config_append = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_dir = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.166828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_config_append = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.167886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_dir = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_file = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.168968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.169872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] publish_errors = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.170986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.171986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] standard_threads = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] publish_errors = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.172939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_journal = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_json = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_stderr = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_syslog = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] standard_threads = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.173889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.173959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.174128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.174212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.174306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] use_journal = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.174388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.174486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] use_json = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.174611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.174686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] use_stderr = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.174830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] use_syslog = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.174907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.175065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:16:14.175141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.175317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.175398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.175554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.175636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.175843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.176979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.177960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.178973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.179946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.180913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.181949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.182882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.183977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.184920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.185932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.186856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.187947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.188974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.189901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.190864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.191025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.191107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.191260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.191337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.191516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.191684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.192924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.193968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.194929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.195954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.196911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.197999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.198945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.external = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.199984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.password = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.token = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.external = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.200945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.password = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] auth.token = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.201901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.202861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.203968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.204975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.205963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.206997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.207259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.207698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.207698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.207832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.207832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.207958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.208998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.209956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.210972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.212998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.213947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.214950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.215913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216149 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.216891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.217987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.218959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.219924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.220988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.221934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.222092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.222169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.222334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.222468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.222575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.222659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.223337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.223769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.223769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.223918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.223990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.224992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.225979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.226909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.227862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.228848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.229955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.230979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.231936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.232914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.233866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.234997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] role.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] role.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.235946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.236948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.237935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.238067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.238185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.238401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.238496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.239953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.240943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.241903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.242999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.243960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.244079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.244178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.244273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.244416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.244503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.244946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.245930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.246977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.247942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.248910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.connection = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.249856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.250965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 03:16:14.251687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.251930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.252183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.252425 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.252675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.252935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.253182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.253422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:16:14.253595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81486) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 03:16:14.280500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.280724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.280964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.282269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.282506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.282734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.290108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.291953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.318420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.318645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.319033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.319269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.372598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.373151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.422250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.422423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.422800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.423039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.580899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.581041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.581137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.581264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.581482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.605973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.606099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.606179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.606313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.606393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.606527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.617562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.617902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.630583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.630838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.630928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.631964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.632042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.632195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81486) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:16:14.649176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.649382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:16:14.649722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.649803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.649920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:16:14.650018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:16:14.650193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.650276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.650452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:16:14.650532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.650736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.650958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.651040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.651220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.651505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.651772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.661155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.661432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.661700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.661926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.662055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.662195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.662475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.662765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.668930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.669759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.671810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.672092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.672409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.672645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.672793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.672948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673351 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.673869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.674958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.675241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.675515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.697172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.698180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.698620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.698904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.699654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.699942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.703534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.703817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.704099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.704631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.704945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.705237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.711147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.711433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.712244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.712534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.715499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.715796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.716077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.716354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.716624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.716760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.716917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.717045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.717175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.717305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.717445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.717575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.717753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.718038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.718316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.718588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.735426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.735706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.736394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.736678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.739620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.740703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.742821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.743117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.743928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.744228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.747574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.748719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.750688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.751879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.753915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.754939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.755478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.755767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.755896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.756067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.756206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.756369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.756498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.756730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.757062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.757355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.773054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.773353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.773622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.773888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.774008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.774311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.774587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.774855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.780704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.781755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.782233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.782511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.783290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.783575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.785125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.785608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.785904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.786198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.786312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.786486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.786633 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.786895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.787193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.787491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.797716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.797999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.798280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.798559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.798820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.798905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.799098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.799188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.799379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.799463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.799665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.799953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.800238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.800530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.814581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.815962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.818301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.819668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.821186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.822573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.822662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.824016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.824108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.824337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.824629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.825552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.825849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.826143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.831057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.833141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.833350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.834831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.836247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.836538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.836855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.837136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.837959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.838256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.838547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.838836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.845007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.845298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.845578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.846785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.847107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.847393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.853143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.853427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.853703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.853972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.854240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.854518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.854787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.854989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81486) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:16:14.855079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.855277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.855369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.855566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.855656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.855839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.855936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.856931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.857218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.857492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.857786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.858060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.858329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.858599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81486) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:16:14.891577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562bf572e9c0 pid: 81485 (default app) Aug 24 03:16:14.894049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562bf572e9c0 pid: 81486 (default app) Aug 24 03:16:14.895601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441dd029-aa30-4a05-a210-f58cded85112 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:14.895881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441dd029-aa30-4a05-a210-f58cded85112 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:14.896144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441dd029-aa30-4a05-a210-f58cded85112 None None] PATH_INFO: `/v3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:14.900286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1/1] 10.222.0.81 () {50 vars in 791 bytes} [Wed Aug 24 03:16:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:21.556867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:21.557560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:21.557560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:21.561810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1/2] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:21.568073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:21.568512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:21.568640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:21.640490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: INFO dbcounter [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] Registered counter for database keystone Aug 24 03:16:21.663199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG oslo_db.sqlalchemy.engines [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81485) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 03:16:21.669564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writer thread running {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 03:16:21.852143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG passlib.handlers.bcrypt [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81485) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 03:16:21.852467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG passlib.handlers.bcrypt [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81485) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 03:16:21.907551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:21.909171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:22.038606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 2/3] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 03:16:22.476990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:22.545591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: INFO dbcounter [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] Registered counter for database keystone Aug 24 03:16:22.567429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG oslo_db.sqlalchemy.engines [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81486) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 03:16:22.573712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writer thread running {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 03:16:22.697603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0b66c-eb6b-4119-abf4-27827bfb53a5 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:22.701954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:22.702582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:22.702864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:22.703137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:22.712736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:22.774150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 03:16:22.774408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f911f-282a-4011-8d9f-b4c5bbae65c6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:22.784858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 2/4] 10.222.0.81 () {58 vars in 1324 bytes} [Wed Aug 24 03:16:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:16:22.791237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:22.806681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6ee08-26f8-46d5-91bc-4878fa97c78a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:22.811116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:22.812215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:22.812500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:22.812825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] PATH_INFO: `/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:22.814371 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] RBAC: Authorizing `identity:get_project(project_id=35c28eab9a294b9ba17b7f5b9c24d868)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:22.869114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 03:16:22.869459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6aab01-75f5-4d12-a985-fef8959fd9d6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:22.874226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 3/5] 10.222.0.81 () {56 vars in 1207 bytes} [Wed Aug 24 03:16:22 2022] GET /identity/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:24.765003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914d288-295f-4a83-bb2e-790ab4d2ef5d None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:24.765546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914d288-295f-4a83-bb2e-790ab4d2ef5d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:24.765546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914d288-295f-4a83-bb2e-790ab4d2ef5d None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:24.769619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 3/6] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:16:24.775845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:24.776274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:24.776392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:24.796354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:24.797882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:24.805851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 4/7] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 03:16:25.074304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914d288-295f-4a83-bb2e-790ab4d2ef5d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:25.087394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914d288-295f-4a83-bb2e-790ab4d2ef5d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:25.091627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f438a75-f4b9-4015-bfe2-14824b0da0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:25.092220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f438a75-f4b9-4015-bfe2-14824b0da0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:25.092500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f438a75-f4b9-4015-bfe2-14824b0da0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:25.092825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f438a75-f4b9-4015-bfe2-14824b0da0f7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:25.100392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f438a75-f4b9-4015-bfe2-14824b0da0f7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:25.102780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f438a75-f4b9-4015-bfe2-14824b0da0f7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:25.111376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 4/8] 10.222.0.81 () {58 vars in 1324 bytes} [Wed Aug 24 03:16:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:16:25.116681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:25.126150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747701c-ad2b-4744-abd9-1ec380e184be None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:25.130505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05995323-efbe-4963-9c7e-4ed6135314a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:25.131122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995323-efbe-4963-9c7e-4ed6135314a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:25.131403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995323-efbe-4963-9c7e-4ed6135314a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:25.131677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995323-efbe-4963-9c7e-4ed6135314a5 admin admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:25.133027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05995323-efbe-4963-9c7e-4ed6135314a5 admin admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:25.134858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05995323-efbe-4963-9c7e-4ed6135314a5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:25.136381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 5/9] 10.222.0.81 () {56 vars in 1198 bytes} [Wed Aug 24 03:16:25 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:27.092355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1ceb-03af-41ae-bb70-f78cb63fa3ab None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.092897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1ceb-03af-41ae-bb70-f78cb63fa3ab None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.092987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1ceb-03af-41ae-bb70-f78cb63fa3ab None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.094262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 5/10] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:27.099450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.099860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.100073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.118579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:27.119781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:27.156375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843eae2-256d-4965-901b-31b3736b0869 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.156651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843eae2-256d-4965-901b-31b3736b0869 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.156971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843eae2-256d-4965-901b-31b3736b0869 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.158294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 6/11] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:16:27.163536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.163769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.164050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.206612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 6/12] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:27.206815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG passlib.handlers.bcrypt [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81486) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 03:16:27.207177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG passlib.handlers.bcrypt [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81486) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 03:16:27.253241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:27.254837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:27.355216 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 7/13] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:16:27.473382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:27.504027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:27.585143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c042c16-b424-40d0-a796-cf6869b96ce5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:27.618462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:27.619077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.619350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.619616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.622625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:27.631326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:27.640266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:27.641385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:27.645769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 7/14] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:27.647160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:27.728460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19203ca9-9af0-4d0c-ac94-8821ba15ba6a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:27.728647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:27.730904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:27.752902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbcae5-2572-4851-af86-e5bbd0dc5c58 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:27.767744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:27.768609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.768926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.769200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.785989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:27.786599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.786880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.787149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.787820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:27.788121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:27.790594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:27.802573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 8/15] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:27.808624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:27.838354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:27.839877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:27.910599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c4e8-ee13-44bb-ad24-8afedb64732a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:27.943848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 8/16] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:16:27.946746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:27.947372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:27.947651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:27.947923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:27.949104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:27.949716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:27.950559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:27.952099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 9/17] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:27 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:27.952477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:27.957436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:27.972941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:28.053820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3148-95b5-4afd-81e3-93a5f88fde78 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:28.059424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b59d26-fdd1-4411-9c29-5704c21ce5e8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:28.089454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:28.090300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:28.090583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:28.090858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:28.095157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:28.095788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:28.096071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:28.096344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:28.097477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:28.098930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:28.100490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 10/18] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:28.106866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:28.110875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:28.112370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:16:28.131466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:28.132369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:16:28.134076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:16:28.134076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:16:28.134076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:28.134076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 9/19] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:16:28.139279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:28.190123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:28.195310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:28.209411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa94c79-0e41-40ef-951b-7634a4b3fcb1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:28.240921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f0f3-8e3c-4b9b-9702-b7ac8bdccf42 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:28.244187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:28.244825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:28.245107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:28.245378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:28.246105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.domain.id=default, user.id=5a8025e93c084f56aeda2462957536b8) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:28.246408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:28.247850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1676c-8b35-420d-9d83-d8ae73dfa4a5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:28.254743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 11/20] 10.222.0.81 () {56 vars in 1322 bytes} [Wed Aug 24 03:16:28 2022] GET /identity/v3/role_assignments?user.id=5a8025e93c084f56aeda2462957536b8&scope.domain.id=default&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:28.277145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:28.277771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:28.278054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:28.278328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:28.278999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:28.279303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:28.280752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45a204-8a32-454f-afcd-4bbe1883d4c8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:28.302520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 10/21] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:30.663035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b6a7c-828d-484f-b5a5-2ace2821832e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:30.663542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b6a7c-828d-484f-b5a5-2ace2821832e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:30.663643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b6a7c-828d-484f-b5a5-2ace2821832e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:30.664969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 12/22] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:30.671385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d3e3b-46b2-4272-965e-cf864f3184e3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:30.671824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d3e3b-46b2-4272-965e-cf864f3184e3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:30.671964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d3e3b-46b2-4272-965e-cf864f3184e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:30.694990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e83d3e3b-46b2-4272-965e-cf864f3184e3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:30.696202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e83d3e3b-46b2-4272-965e-cf864f3184e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:30.782556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 11/23] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:30.806310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004722-1104-4bf4-be96-7fab26b8db82 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:30.806759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004722-1104-4bf4-be96-7fab26b8db82 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:30.807275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004722-1104-4bf4-be96-7fab26b8db82 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:30.809003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 13/24] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:30.818626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d477-af7e-4ac7-909c-4a98d23d7ada None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:30.818912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d477-af7e-4ac7-909c-4a98d23d7ada None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:30.819183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d477-af7e-4ac7-909c-4a98d23d7ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:30.843497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-42d5d477-af7e-4ac7-909c-4a98d23d7ada None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:30.844692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-42d5d477-af7e-4ac7-909c-4a98d23d7ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:30.889657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc8bf-017b-4ad7-a7c1-10210e7201c8 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:30.895661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc8bf-017b-4ad7-a7c1-10210e7201c8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:30.896165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc8bf-017b-4ad7-a7c1-10210e7201c8 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:30.898142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 14/25] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:30.904086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d932-b8d3-40d6-8e95-45152bfd1443 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:30.909391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:30.909769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d932-b8d3-40d6-8e95-45152bfd1443 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:30.910257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d932-b8d3-40d6-8e95-45152bfd1443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:30.943963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9255d932-b8d3-40d6-8e95-45152bfd1443 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:30.946171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9255d932-b8d3-40d6-8e95-45152bfd1443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:30.977008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 12/26] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:31.062467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 15/27] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:31.105152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1ec31-4d88-461e-8e57-2eb023a86a6c None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:31.105436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1ec31-4d88-461e-8e57-2eb023a86a6c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:31.105905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1ec31-4d88-461e-8e57-2eb023a86a6c None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:31.107419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 13/28] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:31.114166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:31.114604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:31.115036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:31.139967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:31.141394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:31.242173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 16/29] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:31.329014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1ec31-4d88-461e-8e57-2eb023a86a6c None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:31.347882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:31.351254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:31.493589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956662-a445-4a65-8cfb-05e9d18845f1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:31.521588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1ec31-4d88-461e-8e57-2eb023a86a6c None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:31.561810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:31.562457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:31.562733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:31.562998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:31.580795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:31.582674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:31.589281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:31.589913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:31.590186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:31.590451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:31.602822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:16:31.603768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:31.604899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:16:31.605891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:16:31.605891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:16:31.605891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:31.607253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 17/30] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:16:31.608148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:31.609793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:31.610799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:31.611734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:31.615888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 14/31] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:31.616776 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:31.657855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:31.692026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:31.733818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109aec4-8915-40b4-9ce5-049403177b8b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:31.758638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2add3b-7dfa-429e-a703-837d704e059b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:31.772082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:31.772893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:31.773559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:31.773708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:31.774546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:31.776794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:31.803617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:31.804478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:31.804915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:31.805319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:31.806126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:31.807863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:31.916936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 15/32] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Aug 24 03:16:31 2022] POST /identity/v3/roles => generated 225 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:31.918181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:31.944239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 18/33] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Aug 24 03:16:31 2022] POST /identity/v3/roles => generated 219 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:31.950147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:32.141598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009753cc-eb3e-462d-b378-c5ce349b1d51 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.166315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:32.175330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:32.181602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f2070-d5c4-4813-acf3-da3791bb681b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.184824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.185734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.186233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.186694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.187569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:32.188059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.191432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304907-1feb-4d17-9ddc-91037c3b366d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.203494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 19/34] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:32.205278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fee85-f81c-4ede-9803-3a78cbb6a384 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.205756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fee85-f81c-4ede-9803-3a78cbb6a384 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.206219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fee85-f81c-4ede-9803-3a78cbb6a384 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.207740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 20/35] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:32.210803 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fee85-f81c-4ede-9803-3a78cbb6a384 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:32.226651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.227144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.227432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.227712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.228434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:32.228842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.230653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb81a-de19-4d7b-b2b2-a4c0837b928d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.245678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 16/36] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:32.246985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.247274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.247552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.282752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:32.284493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:32.347883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fee85-f81c-4ede-9803-3a78cbb6a384 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.374272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 17/37] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:32.375265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:32.387504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.388564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.389065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.389525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.391047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.392745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.394694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 21/38] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:32 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:32.401856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:32.453565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:32.502112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:32.522135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9b786-2155-4294-8673-1aee7d8d03fd None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.567357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b61c2-43e0-4b17-945c-300ef8200ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.568483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b61c2-43e0-4b17-945c-300ef8200ff1 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.568997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b61c2-43e0-4b17-945c-300ef8200ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.569276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b61c2-43e0-4b17-945c-300ef8200ff1 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.570217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b61c2-43e0-4b17-945c-300ef8200ff1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.571725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b61c2-43e0-4b17-945c-300ef8200ff1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.607099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 18/39] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:16:32 2022] POST /identity/v3/projects => generated 301 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:32.608598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40170d6f-031c-44fe-80a1-343539a31027 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.608919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40170d6f-031c-44fe-80a1-343539a31027 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.609198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40170d6f-031c-44fe-80a1-343539a31027 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.610560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 19/40] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:32.619535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.621549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.621938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.639080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969512d-0168-4f7f-8587-7b961455d879 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.649914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:32.652992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:32.682626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.683748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.684164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.684572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.686726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.688989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.691854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 22/41] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:32.693077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:32.742461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 20/42] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:32.743325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:32.832352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:32.834335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275b426-81ba-447e-8889-230b6f28e0dc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.854772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:32.872319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d97923-e43e-4714-be73-17aa21ca1e07 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:32.881407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.882029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.882311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.882593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.883259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.884751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.904508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 23/43] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Aug 24 03:16:32 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:32.918403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:32.919000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:32.919350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:32.919630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] PATH_INFO: `/v3/domains/default/users/5a8025e93c084f56aeda2462957536b8/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:32.922363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5a8025e93c084f56aeda2462957536b8, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:32.923864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:32.940646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 21/44] 10.222.0.81 () {58 vars in 1361 bytes} [Wed Aug 24 03:16:32 2022] PUT /identity/v3/domains/default/users/5a8025e93c084f56aeda2462957536b8/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:16:33.032849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:33.149682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf628f-ad55-4520-bdc3-393eac4516d0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:33.183468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a75e8-b42c-4d31-b4a2-aad64a7bfdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:33.184025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a75e8-b42c-4d31-b4a2-aad64a7bfdfa None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:33.184304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a75e8-b42c-4d31-b4a2-aad64a7bfdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:33.184576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a75e8-b42c-4d31-b4a2-aad64a7bfdfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:33.185717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a75e8-b42c-4d31-b4a2-aad64a7bfdfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:33.187180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a75e8-b42c-4d31-b4a2-aad64a7bfdfa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:33.188735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 24/45] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:33.194202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:33.295782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2014f3-6a37-40f3-aeac-135840e2ac09 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:33.322105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:33.332451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86308a50-01fc-4c05-a5e5-3499ced2b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:33.333079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86308a50-01fc-4c05-a5e5-3499ced2b1c3 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:33.333358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86308a50-01fc-4c05-a5e5-3499ced2b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:33.333631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86308a50-01fc-4c05-a5e5-3499ced2b1c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:33.334286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86308a50-01fc-4c05-a5e5-3499ced2b1c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:33.335739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86308a50-01fc-4c05-a5e5-3499ced2b1c3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:33.362833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 22/46] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:16:33 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:16:34.882215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70923fa4-03c5-4921-b540-7f241cd65113 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:34.882727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70923fa4-03c5-4921-b540-7f241cd65113 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:34.882807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70923fa4-03c5-4921-b540-7f241cd65113 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:34.884117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 25/47] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:34.890288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b66ed-770e-444a-99e2-8b5fa42f9ef6 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:34.890683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b66ed-770e-444a-99e2-8b5fa42f9ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:34.890841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b66ed-770e-444a-99e2-8b5fa42f9ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:34.909572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-367b66ed-770e-444a-99e2-8b5fa42f9ef6 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:34.910615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-367b66ed-770e-444a-99e2-8b5fa42f9ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:34.977701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 23/48] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:16:35.254980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70923fa4-03c5-4921-b540-7f241cd65113 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.290597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acc217-c889-4e43-8f7a-a04abb45b78a None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.290883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acc217-c889-4e43-8f7a-a04abb45b78a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.291153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acc217-c889-4e43-8f7a-a04abb45b78a None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.292702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 24/49] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:16:35.297796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.298082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.298362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.302214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:35.316908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:35.318119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:35.364907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70923fa4-03c5-4921-b540-7f241cd65113 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:35.388414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 25/50] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:16:35.400206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:35.400865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.401148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.401942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.411917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:35.416604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:35.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:35.418918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 26/51] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:35.424305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.465686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:35.524464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01accf20-b14e-409f-bb33-beeaea53baa1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:35.560058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:35.560663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.561042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.561320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.561992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:35.562297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:35.563688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:35.576089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 26/52] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:35.581831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.660649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.684680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589ccaf-c612-4896-b6de-18f72f781527 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:35.686313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:35.721109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:35.721766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.722048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.722324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.723538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:35.725090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:35.726767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 27/53] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:35 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:35.733547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.838284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852f38d-9e09-4804-a2c8-7e27d84ba536 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:35.873212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7355de-066a-4b08-9058-47e8b11d25b4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:35.874727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:35.877993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:35.878623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.878902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.879175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.880305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:35.881788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:35.883536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 28/54] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:35.890048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.915013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:35.915645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:35.915930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:35.916202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:35.917389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:35.918923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:35.920506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 27/55] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:35.925984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:35.930374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:35.991349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003c3a0-3293-4d65-a2d4-9f3501b9dbc1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:36.025276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:36.025875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:36.026037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc4ad-af16-4f2b-bc16-5376b0160702 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:36.026157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:36.026437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:36.028013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.domain.id=default, user.id=5a8025e93c084f56aeda2462957536b8) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:36.028327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:36.029813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645eafb-b9db-4780-a9e7-29fc2d53c5ff None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:36.035687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 29/56] 10.222.0.81 () {56 vars in 1322 bytes} [Wed Aug 24 03:16:35 2022] GET /identity/v3/role_assignments?user.id=5a8025e93c084f56aeda2462957536b8&scope.domain.id=default&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:36.062265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3c5fe-c5e7-4b75-a0c3-1dd387fa23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:36.062894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c5fe-c5e7-4b75-a0c3-1dd387fa23c4 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:36.063176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c5fe-c5e7-4b75-a0c3-1dd387fa23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:36.063451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c5fe-c5e7-4b75-a0c3-1dd387fa23c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:36.064119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3c5fe-c5e7-4b75-a0c3-1dd387fa23c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:36.065637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3c5fe-c5e7-4b75-a0c3-1dd387fa23c4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:36.097132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 28/57] 10.222.0.81 () {60 vars in 1140 bytes} [Wed Aug 24 03:16:35 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:16:38.017376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76044-3fd1-44f6-97b3-b327783a44a2 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:38.017875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76044-3fd1-44f6-97b3-b327783a44a2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:38.017986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f76044-3fd1-44f6-97b3-b327783a44a2 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:38.019256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 30/58] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:38.024871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:38.025264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:38.025428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:38.043911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:38.045158 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:38.061658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:38.112237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 29/59] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:38.395072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76044-3fd1-44f6-97b3-b327783a44a2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:38.495230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f76044-3fd1-44f6-97b3-b327783a44a2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:38.497530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:38.531058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c136bd-ebcc-42c1-8bff-28a029139db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:38.531677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c136bd-ebcc-42c1-8bff-28a029139db1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:38.531961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c136bd-ebcc-42c1-8bff-28a029139db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:38.532237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c136bd-ebcc-42c1-8bff-28a029139db1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:38.533370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c136bd-ebcc-42c1-8bff-28a029139db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:38.534833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c136bd-ebcc-42c1-8bff-28a029139db1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:38.536396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 31/60] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:38.542120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:38.640587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20fc93-c4bc-4d61-b255-a27a3b53f99e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:38.681741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95ce02-63ce-4bd7-bc41-0f76328e590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:38.682279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95ce02-63ce-4bd7-bc41-0f76328e590d None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:38.682562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95ce02-63ce-4bd7-bc41-0f76328e590d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:38.682839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95ce02-63ce-4bd7-bc41-0f76328e590d None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:38.683497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95ce02-63ce-4bd7-bc41-0f76328e590d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:38.685009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95ce02-63ce-4bd7-bc41-0f76328e590d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:38.712820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 30/61] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:16:38 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:16:41.815024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf21ea-af52-4017-b5cb-dcdf80bad525 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:41.815563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf21ea-af52-4017-b5cb-dcdf80bad525 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:41.815563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf21ea-af52-4017-b5cb-dcdf80bad525 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:41.816405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db98ae-c141-4ce4-948c-4912a6aca612 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:41.816963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 32/62] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:41.817513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db98ae-c141-4ce4-948c-4912a6aca612 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:41.817802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db98ae-c141-4ce4-948c-4912a6aca612 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:41.819437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 31/63] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:16:41.819542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c374c-9bb7-4b96-9fe8-39b4b08eb55a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:41.819856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c374c-9bb7-4b96-9fe8-39b4b08eb55a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:41.820143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c374c-9bb7-4b96-9fe8-39b4b08eb55a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:41.821690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 33/64] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:41.824665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:41.825374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:41.825877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:41.826577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75cb8a-90a1-4a11-a13b-e09911c04807 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:41.827577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75cb8a-90a1-4a11-a13b-e09911c04807 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:41.827739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75cb8a-90a1-4a11-a13b-e09911c04807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:41.852147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3a75cb8a-90a1-4a11-a13b-e09911c04807 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:41.852759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:41.854044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:41.854381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3a75cb8a-90a1-4a11-a13b-e09911c04807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:41.856304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:41.947905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 34/65] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:16:41.953236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ff298-9976-48f3-8d16-b39b4eb117a1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:41.953236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ff298-9976-48f3-8d16-b39b4eb117a1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:41.953236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ff298-9976-48f3-8d16-b39b4eb117a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:41.960480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 32/66] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:41.981065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b50ff298-9976-48f3-8d16-b39b4eb117a1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:41.982250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b50ff298-9976-48f3-8d16-b39b4eb117a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:42.074635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 35/67] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:42.300054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.314404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e52ae-1425-448f-96af-f9dd750a1e76 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.314711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e52ae-1425-448f-96af-f9dd750a1e76 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.314954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e52ae-1425-448f-96af-f9dd750a1e76 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.316293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 36/68] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:42.322409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.322693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.322974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.349903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:42.351012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:42.387810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:42.442358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 37/69] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:42.443218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.443573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5762-60e4-4211-b5c1-88b3f6439559 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:42.482813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecb397-c507-4eab-8170-d2a129c0e8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:42.483954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecb397-c507-4eab-8170-d2a129c0e8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.484240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecb397-c507-4eab-8170-d2a129c0e8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.484513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecb397-c507-4eab-8170-d2a129c0e8b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.492085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:42.496050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecb397-c507-4eab-8170-d2a129c0e8b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-c2ecb397-c507-4eab-8170-d2a129c0e8b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:42.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:42.503333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:42.504099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:42.512386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 33/70] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:42.513961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2039-c9c0-4ebe-867f-1c87494884ec None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.514249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2039-c9c0-4ebe-867f-1c87494884ec None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.514548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2039-c9c0-4ebe-867f-1c87494884ec None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.515961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 34/71] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:42.516541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca2039-c9c0-4ebe-867f-1c87494884ec None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.562940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473898c3-c7b4-4233-946c-bb6caea6ca87 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:42.600073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:42.600626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.600977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.601251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.612457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:42.617446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:42.618265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:42.619011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 38/72] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:42.619622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.630114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca2039-c9c0-4ebe-867f-1c87494884ec None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:42.645699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:42.667144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73483920-c396-4118-9294-41d508f4c830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:42.667770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73483920-c396-4118-9294-41d508f4c830 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.668053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73483920-c396-4118-9294-41d508f4c830 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.668326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73483920-c396-4118-9294-41d508f4c830 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.678980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73483920-c396-4118-9294-41d508f4c830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-73483920-c396-4118-9294-41d508f4c830 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:42.683681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:42.684533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:42.685312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 35/73] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:42.685967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.686858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.686858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.704893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:42.706108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:42.737903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81218bf5-1883-4979-b356-30c3eb82c907 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:42.760755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:42.773143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:42.773743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.774022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.774292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.774958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:42.775263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:42.776740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:42.777709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 36/74] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:42.780922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.791919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 39/75] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:42.792996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.883476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:42.896036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8143ebb-1a96-4890-b7da-9ba23700e366 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:42.910935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157b97-499c-418e-b757-96a66dc4636e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:42.934154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:42.934917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.935201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.935477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.936143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:42.936442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:42.938060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:42.951052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 37/76] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:42.951052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:42.952377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:42.953015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:42.953299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:42.953577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:42.954366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:42.954672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:42.956195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:42.970502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 40/77] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:42.970502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.008490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.061546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb872ea-fb9a-4b90-b5dc-c59b14ac24e4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.077877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153643-6b01-45d8-8a9e-8a42c8e5274e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.096170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.096813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.097099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.097376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.107298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:43.112734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:43.113570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:43.114348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 38/78] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:43.115047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.134041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.134580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.134856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.135130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.149820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.151279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.152960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 41/79] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:42 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:43.153720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.160623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.226956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d79b6-1ee8-4014-be0c-8f5e73a63c0a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.237797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.258585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532529-effe-4f2b-baba-e914cd7ac60f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.262187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.262814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.263098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.263375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.264525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.266575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.268171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 39/80] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:43.269604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.294385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.295045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.295327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.295605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.296828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.298284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.299874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 42/81] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:43.300633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.369913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.371759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309653-54c2-4c3f-9f52-c15b63124ce8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.403033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5c557-d8a4-49d7-a239-f39b7e7be1e8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.406124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.406761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.407036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.407311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.408447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.409946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.411524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 40/82] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:43.412297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.439711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.440423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.440802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.441093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.441789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:43.442094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.443487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.451563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 43/83] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:43.452951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.460342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.515459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688d847-de24-4c24-a25d-e2f558a62bd1 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.548516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.549138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.549436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.549707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.553766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea6d3e-b8d3-4877-aea3-f54d798b38b8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.582729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.588121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.588721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.589098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.589375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.618606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.620104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.657341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.658763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.666410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.721576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 41/84] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:43.722419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.761233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 44/85] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:43.762066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:43.793650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.821518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2d96-7112-47df-ae18-61d843a48c8a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.855880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.856483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.856756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.857054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.863048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac2e7f-bbe6-4ad5-94cc-407ebc484a16 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:43.872919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:43.903397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:43.904032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:43.904297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:43.904575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:43.905255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.907677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.930757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:43.932419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:43.936839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 45/86] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:16:43 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:43.937680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.045942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.049320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 42/87] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:44.050213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.067858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5cb13-7349-4e2f-b242-3b824ac9455b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.104355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.109651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.110503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.110934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.111387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.112559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.114081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.115681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 46/88] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:43 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.116461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.176427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1367e7-2abe-4226-a475-38e1a133752c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.218065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.219204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.219719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.220198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.221734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.223401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.225458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 43/89] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.226452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.250880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb6ff-9a31-4b4d-85d8-d495aa7a5857 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.293637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.294279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.294564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.294853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.296102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.298352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.300332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 47/90] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.301154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.340888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.357417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0e491-3434-4af6-b37c-5c952eca6f8d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.395625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.396712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.397292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.397771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.399129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.404916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.405766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.408903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 44/91] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.414749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.437112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88409a-5dd6-42e2-84a7-373b884ff974 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.478757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.479380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.479666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.479943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.538337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862efda-e239-48f1-9f23-244fceb73cd7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.556191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.557699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.577400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.578444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.578979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.579446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.580433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] RBAC: Adding query filter params (role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=043ada6c6c4a47fdac79dc17270a1821) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:44.582529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.583731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.585602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.595166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 45/92] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/role_assignments?user.id=043ada6c6c4a47fdac79dc17270a1821&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.596277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.669113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 48/93] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:44.669320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.669936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.717314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0744-2299-4ffc-abfd-e7da075b52fc None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.766255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.767359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.767844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.768338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.769416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=5a8025e93c084f56aeda2462957536b8) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:44.769994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.771777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.778630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 46/94] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/role_assignments?user.id=5a8025e93c084f56aeda2462957536b8&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.779975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:44.791022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f33681-8de1-4dce-b87c-9065a97677db None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.818785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.833242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.834297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.834792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.835261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.836529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=043ada6c6c4a47fdac79dc17270a1821) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:44.837038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.838524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.845135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 49/95] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/role_assignments?user.id=043ada6c6c4a47fdac79dc17270a1821&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=fed56dd461254bc682cf1955076ea721 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.911144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f9d35-27a9-4707-9a09-ea0796b7c892 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:44.924262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:44.953787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c67441-7b72-45bf-8de1-a95f35fa3647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:44.954462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c67441-7b72-45bf-8de1-a95f35fa3647 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:44.954722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c67441-7b72-45bf-8de1-a95f35fa3647 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:44.955003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c67441-7b72-45bf-8de1-a95f35fa3647 None admin] PATH_INFO: `/v3/projects/050ae3d43d0140c78af4692911503956` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:44.956335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c67441-7b72-45bf-8de1-a95f35fa3647 None admin] RBAC: Authorizing `identity:get_project(project_id=050ae3d43d0140c78af4692911503956)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:44.957914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c67441-7b72-45bf-8de1-a95f35fa3647 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:44.960087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 47/96] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/projects/050ae3d43d0140c78af4692911503956 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:44.966058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:45.098339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857224d-3ba5-404c-9184-fb4f0d97dacf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:45.141203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:45.141831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:45.142122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:45.142395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:45.143131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=050ae3d43d0140c78af4692911503956, user.id=043ada6c6c4a47fdac79dc17270a1821) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:45.143438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:45.144980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b38439-8ba2-4832-be36-2d5043231610 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:45.150893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 50/97] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:44 2022] GET /identity/v3/role_assignments?user.id=043ada6c6c4a47fdac79dc17270a1821&scope.project.id=050ae3d43d0140c78af4692911503956&role.id=fed56dd461254bc682cf1955076ea721 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:46.521754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20898c98-09ea-41e2-82ab-f9f5e36fb875 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:46.522260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20898c98-09ea-41e2-82ab-f9f5e36fb875 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:46.522332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20898c98-09ea-41e2-82ab-f9f5e36fb875 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:46.523814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 48/98] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:46.529217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fc8f5-dc9e-4dfe-b304-8814278f2da4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:46.529611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fc8f5-dc9e-4dfe-b304-8814278f2da4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:46.529790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fc8f5-dc9e-4dfe-b304-8814278f2da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:46.554254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fa8fc8f5-dc9e-4dfe-b304-8814278f2da4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:46.555520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fa8fc8f5-dc9e-4dfe-b304-8814278f2da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:46.616937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:46.639192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 51/99] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:46.904197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e27f-cedc-4182-bad7-596184a489c9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:46.907189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e27f-cedc-4182-bad7-596184a489c9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:46.907189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e27f-cedc-4182-bad7-596184a489c9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:46.907189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 49/100] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:46.911932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:46.912352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:46.912734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:46.937584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:46.939331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:47.019900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 52/101] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:47.023483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e27f-cedc-4182-bad7-596184a489c9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:47.145447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e27f-cedc-4182-bad7-596184a489c9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:47.190081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:47.190653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.190936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.191213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.207877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:47.213133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:47.213967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:47.214808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 50/102] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:47.222312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:47.347266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0cb74-fc36-4859-b71b-adaca25b41b3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:47.369270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:47.392267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:47.392926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.393218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.393554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.394238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:47.394544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:47.396001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:47.404963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 53/103] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:47.411309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:47.456891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:47.464088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:47.598539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf971c17-cd41-40c4-b467-4d55f4d13b81 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:47.663674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7859de3-82b3-4017-917b-f9da87cd70db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:47.664565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7859de3-82b3-4017-917b-f9da87cd70db None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.664839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ff7cd-0b3b-4f31-b3e1-a0ea41f62271 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:47.665527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7859de3-82b3-4017-917b-f9da87cd70db None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.665948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7859de3-82b3-4017-917b-f9da87cd70db None admin] PATH_INFO: `/v3/users/51db1f1a5cae4aeeba8362c7411fdbc0` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.682801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7859de3-82b3-4017-917b-f9da87cd70db None admin] RBAC: Authorizing `identity:get_user(user_id=51db1f1a5cae4aeeba8362c7411fdbc0)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:47.684491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7859de3-82b3-4017-917b-f9da87cd70db None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:47.686958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 51/104] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:47 2022] GET /identity/v3/users/51db1f1a5cae4aeeba8362c7411fdbc0 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:47.688882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b3d3-15b0-41aa-9e46-7d82dfba8a5d None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.689312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b3d3-15b0-41aa-9e46-7d82dfba8a5d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.689730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b3d3-15b0-41aa-9e46-7d82dfba8a5d None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.691360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 52/105] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:47.696474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe2ef-cfdc-432b-89ce-dc982eccbea4 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.697227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe2ef-cfdc-432b-89ce-dc982eccbea4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.697734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe2ef-cfdc-432b-89ce-dc982eccbea4 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.699455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 53/106] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:47.700374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe2ef-cfdc-432b-89ce-dc982eccbea4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:47.712396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b6b77-7714-4cc4-a3a3-e55f773fa2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:47.712998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b6b77-7714-4cc4-a3a3-e55f773fa2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.713280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b6b77-7714-4cc4-a3a3-e55f773fa2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.713558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b6b77-7714-4cc4-a3a3-e55f773fa2c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.715140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b6b77-7714-4cc4-a3a3-e55f773fa2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:47.717328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b6b77-7714-4cc4-a3a3-e55f773fa2c1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:47.719191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 54/107] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:16:47.721036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce48d1-fc5e-46e5-bc9a-8b8811b2218d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.721321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce48d1-fc5e-46e5-bc9a-8b8811b2218d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.721601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce48d1-fc5e-46e5-bc9a-8b8811b2218d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.745710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-60ce48d1-fc5e-46e5-bc9a-8b8811b2218d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:47.746932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-60ce48d1-fc5e-46e5-bc9a-8b8811b2218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:47.794744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:47.831172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:47.839407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe2ef-cfdc-432b-89ce-dc982eccbea4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:47.841019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 55/108] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:47.842545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.843053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.843541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.873842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:47.875374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:47.879773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d94ef5-97ca-4369-834a-ddf8da69f179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:47.880628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94ef5-97ca-4369-834a-ddf8da69f179 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:47.881061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94ef5-97ca-4369-834a-ddf8da69f179 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:47.881714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94ef5-97ca-4369-834a-ddf8da69f179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:47.973916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d94ef5-97ca-4369-834a-ddf8da69f179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:47.975720 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d94ef5-97ca-4369-834a-ddf8da69f179 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:48.016992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 56/109] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:48.016992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.110934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:48.111406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 54/110] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:48.112948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879461a-cec3-47e7-9062-bdb1aac50197 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.113432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879461a-cec3-47e7-9062-bdb1aac50197 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.113924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879461a-cec3-47e7-9062-bdb1aac50197 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.115944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 55/111] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:48.117284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e33510-998f-47b2-aec4-8cbd493b7b5f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.117759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e33510-998f-47b2-aec4-8cbd493b7b5f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.118224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e33510-998f-47b2-aec4-8cbd493b7b5f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.119673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 56/112] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:48.120800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e33510-998f-47b2-aec4-8cbd493b7b5f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.155057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ee8c3-c22f-4cb5-8bbf-40ee576bf1d9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.156984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:48.195021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e6585-33e6-4917-85ca-a5ba8af184fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.195913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e6585-33e6-4917-85ca-a5ba8af184fe None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.196332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e6585-33e6-4917-85ca-a5ba8af184fe None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.196839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e6585-33e6-4917-85ca-a5ba8af184fe None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.197678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e6585-33e6-4917-85ca-a5ba8af184fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:48.199314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e6585-33e6-4917-85ca-a5ba8af184fe None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:48.231431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 57/113] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:16:48 2022] POST /identity/v3/projects => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:48.233198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.233741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.234214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.252321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e33510-998f-47b2-aec4-8cbd493b7b5f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.257189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:48.258895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:48.297553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e6459-8eff-4242-aa8c-fd894edcdcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.298922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e6459-8eff-4242-aa8c-fd894edcdcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.299208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e6459-8eff-4242-aa8c-fd894edcdcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.299487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e6459-8eff-4242-aa8c-fd894edcdcd8 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.300904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e6459-8eff-4242-aa8c-fd894edcdcd8 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:48.302711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e6459-8eff-4242-aa8c-fd894edcdcd8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:48.310657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 57/114] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:48.310657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.310657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.310657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.327195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:48.328853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:48.353725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 58/115] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:48.356505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.413897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:48.425651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 58/116] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:48.426746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.460837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:48.488695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8c0ee-6f82-4c77-a849-fc7bc8a7ff7c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.529208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.530194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.530645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.531236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.542908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:48.548518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:48.549559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:48.550332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:48.552264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 59/117] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:48.553240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.555717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2647e-900f-4556-9110-0debe8f89ccc None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.605961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.606589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.606829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.607102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.607837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] RBAC: Adding query filter params (role.id=ed708940f2d64bfab650ce5deae860cd, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=51db1f1a5cae4aeeba8362c7411fdbc0) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:48.608140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:48.609612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:48.626197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 59/118] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/role_assignments?user.id=51db1f1a5cae4aeeba8362c7411fdbc0&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=ed708940f2d64bfab650ce5deae860cd => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:48.627821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.735026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdc140-ee11-4887-b93b-4f454271b3da None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.759048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:48.780395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.781185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.781475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.781756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.795353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:48.801810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:48.804045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 60/119] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:48.806101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.811336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4cf6e-c1cf-4d7f-98d3-946cec6bd188 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.814554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:48.854946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.855558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.855838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.856111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.856805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:48.857109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:48.858625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:48.868182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 60/120] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:48.869224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:48.934780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074157e7-c400-41f9-bd61-4b98374e4f7d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.971564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:48.972424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:48.972868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:48.973351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:48.983800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:48.989121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:48.990136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:48.991246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:48.991802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e007c38-b667-43dd-b096-528d8b862fb1 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:48.993159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 61/121] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:48.994115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.019889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.036806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.037454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.037740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.038142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.049766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:49.057570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:49.058626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 61/122] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:49.059625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.072179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.115898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bcec1-133b-448b-b7f9-8842187e77f3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.152744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.153597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.154013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.154421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.155247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:49.155683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.157440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.167068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 62/123] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.168171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.199203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d420-2cb0-4d4d-898a-4b0b4584a619 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.237252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.237885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.238171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.238448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.239603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.241102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.243085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 62/124] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.243971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.280876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f7fcd-1b95-428e-aefe-67c86e479af8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.297904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.310249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.321350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.322240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.322692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.323137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.324106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:49.324582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.326272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.335759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 63/125] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.336785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.364232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70237a1d-d9f1-48bc-9983-e654f3f3a896 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.401321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.402227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.402500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.402857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.403610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] RBAC: Adding query filter params (name=member) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:49.403991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.405612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.416875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 63/126] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.421563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.456556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de866-1dc0-4c59-a22c-dda06bc1d87f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.498194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.499439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.500301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.501056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.503277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.505833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.509218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 64/127] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.511180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.553247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b229e2d-fb3e-40a6-af04-8e036719bded None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.565027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.587378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.602009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.603154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.604490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.605099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.606459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.608526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.612200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 64/128] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.613683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.637552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd76917-fa4a-49a3-9f96-a937c851dd8d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.682294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.683566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.684064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.684781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.686799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.689150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.691462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 65/129] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.692391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.752292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b18c-c3c1-4414-87a3-6d0aaac17059 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.792295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.793437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.794295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.794739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.796595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.798780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.800616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 65/130] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.801522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.820802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb684-d9f9-4b7e-a812-07e9d6f890b7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.847255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.861495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:49.862292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:49.862706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:49.863087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] PATH_INFO: `/v3/projects/050ae3d43d0140c78af4692911503956` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:49.864420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] RBAC: Authorizing `identity:get_project(project_id=050ae3d43d0140c78af4692911503956)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:49.865972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:49.867535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:49.868127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 66/131] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/projects/050ae3d43d0140c78af4692911503956 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:49.872722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:49.948813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0b4ed-b99c-4f35-afc0-aa7e4e14ac9b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.993862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1101983-3983-4c52-9a15-785dd4c21251 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:49.997226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:50.001546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.002945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.003371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/5a8025e93c084f56aeda2462957536b8/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.006391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b7cc325ca0144afbe3094beaaf32144, user_id=5a8025e93c084f56aeda2462957536b8, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:50.008577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:50.028092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 66/132] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:16:49 2022] PUT /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/5a8025e93c084f56aeda2462957536b8/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:16:50.029515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:50.034643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:50.034643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.034643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.035233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.036672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:50.038663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:50.040842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 67/133] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:49 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:50.046753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:50.118682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:50.170684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982439-e27f-4944-9f7f-53e9223f2d4e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:50.184431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fc110-f2b4-4414-a45a-f2fd9a535c98 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:50.193984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:50.213624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:50.214930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.215411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.216053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] PATH_INFO: `/v3/projects/050ae3d43d0140c78af4692911503956/users/043ada6c6c4a47fdac79dc17270a1821/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.218632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=050ae3d43d0140c78af4692911503956, user_id=043ada6c6c4a47fdac79dc17270a1821, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:50.221214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:50.231921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:50.231921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.231921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.232260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.233357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:50.234936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:50.237060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 67/134] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:50 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:50.238144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:50.243632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 68/135] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:16:50 2022] PUT /identity/v3/projects/050ae3d43d0140c78af4692911503956/users/043ada6c6c4a47fdac79dc17270a1821/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:16:50.244856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:50.363878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61239c63-3cdd-44b4-840d-9640434643b9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:50.373843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b24b1-7674-4f70-84c2-13ee0645ddb0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:50.398986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:50.408522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d716ed-f401-46cb-a4db-00235867245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:50.410545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d716ed-f401-46cb-a4db-00235867245b None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.411233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d716ed-f401-46cb-a4db-00235867245b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.411684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d716ed-f401-46cb-a4db-00235867245b None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/043ada6c6c4a47fdac79dc17270a1821/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.414368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d716ed-f401-46cb-a4db-00235867245b None admin] RBAC: Authorizing `identity:create_grant(project_id=1b7cc325ca0144afbe3094beaaf32144, user_id=043ada6c6c4a47fdac79dc17270a1821, role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:50.416044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d716ed-f401-46cb-a4db-00235867245b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:50.419296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43987-a3b6-4f46-898b-5974564340a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:50.420113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43987-a3b6-4f46-898b-5974564340a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.420370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43987-a3b6-4f46-898b-5974564340a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.420718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43987-a3b6-4f46-898b-5974564340a6 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/043ada6c6c4a47fdac79dc17270a1821/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.423306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43987-a3b6-4f46-898b-5974564340a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b7cc325ca0144afbe3094beaaf32144, user_id=043ada6c6c4a47fdac79dc17270a1821, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:50.426731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43987-a3b6-4f46-898b-5974564340a6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:50.438504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 68/136] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:16:50 2022] PUT /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/043ada6c6c4a47fdac79dc17270a1821/roles/7eef59c95b8945768aab3374bc2721f1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:16:50.442828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 69/137] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:16:50 2022] PUT /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/043ada6c6c4a47fdac79dc17270a1821/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:16:50.578346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a9ff-0fc9-4343-96de-b76df1873738 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.578873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a9ff-0fc9-4343-96de-b76df1873738 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.579287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a9ff-0fc9-4343-96de-b76df1873738 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.581293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 69/138] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:50.587195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e724f1-5852-4ad9-9c91-102c2b5aee9e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:50.587505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e724f1-5852-4ad9-9c91-102c2b5aee9e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:50.587757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e724f1-5852-4ad9-9c91-102c2b5aee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:50.615263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e6e724f1-5852-4ad9-9c91-102c2b5aee9e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:50.616871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e6e724f1-5852-4ad9-9c91-102c2b5aee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:50.703371 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 70/139] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:16:51.034542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361a9ff-0fc9-4343-96de-b76df1873738 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:51.081863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:51.154865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468fda-60ac-4cc6-8d85-a8e6d5b77b5e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:51.155980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468fda-60ac-4cc6-8d85-a8e6d5b77b5e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:51.156656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468fda-60ac-4cc6-8d85-a8e6d5b77b5e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:51.159233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 71/140] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:16:51.165079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:51.165769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:51.166239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:51.172799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361a9ff-0fc9-4343-96de-b76df1873738 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:51.206552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:51.208262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:51.218090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:51.218718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:51.219016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:51.219292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:51.220402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:51.224382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:51.224382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 70/141] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:51.232806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:51.362315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 72/142] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:51.369911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfc64e-aab3-4605-b902-4fc61d12b84f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:51.371626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:51.415036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:51.415646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:51.415926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:51.416206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:51.416881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:51.418325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:51.448221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 71/143] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:16:51 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:16:51.763760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:51.819389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:51.897675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35255d2a-cf1c-4c1c-b5af-62985ebf9d3a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:51.953502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:51.954174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:51.954513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:51.954786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:51.971690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:51.976366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:51.977284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:51.978070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 73/144] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:51.983463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:52.118170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8638e-3cce-4eae-aedc-e3b89e6e3640 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:52.163644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:52.164251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:52.164536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:52.164836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:52.165524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:52.165832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:52.167218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:52.176585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 72/145] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:52.182053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:52.335487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:52.339463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b38968-ad4b-4f58-a13f-919f7c9b4e22 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:52.383518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:52.384069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:52.384354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:52.384609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] PATH_INFO: `/v3/users/51db1f1a5cae4aeeba8362c7411fdbc0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:52.385792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=51db1f1a5cae4aeeba8362c7411fdbc0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:52.387265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:52.389051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 74/146] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:52 2022] GET /identity/v3/users/51db1f1a5cae4aeeba8362c7411fdbc0 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:52.394491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:52.429572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:52.531138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a250fe-0ad3-4805-8ea8-e7e8934d0de1 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:52.568456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe7f33-38b5-4f6f-b0d0-ec007364d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:52.569123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe7f33-38b5-4f6f-b0d0-ec007364d654 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:52.569405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe7f33-38b5-4f6f-b0d0-ec007364d654 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:52.569682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe7f33-38b5-4f6f-b0d0-ec007364d654 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:52.570824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe7f33-38b5-4f6f-b0d0-ec007364d654 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:52.572289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe7f33-38b5-4f6f-b0d0-ec007364d654 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:52.574119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 73/147] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:52 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:52.579735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:52.691239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11452e65-c464-4433-ab1a-b7463a29d1a7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:52.731256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158d8c9-45fc-4f53-a910-462b00279621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:52.731917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158d8c9-45fc-4f53-a910-462b00279621 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:52.732194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158d8c9-45fc-4f53-a910-462b00279621 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:52.732465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158d8c9-45fc-4f53-a910-462b00279621 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/51db1f1a5cae4aeeba8362c7411fdbc0/roles/ed708940f2d64bfab650ce5deae860cd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:52.734400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158d8c9-45fc-4f53-a910-462b00279621 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b7cc325ca0144afbe3094beaaf32144, user_id=51db1f1a5cae4aeeba8362c7411fdbc0, role_id=ed708940f2d64bfab650ce5deae860cd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:52.735871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158d8c9-45fc-4f53-a910-462b00279621 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:52.761693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 75/148] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:16:52 2022] PUT /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144/users/51db1f1a5cae4aeeba8362c7411fdbc0/roles/ed708940f2d64bfab650ce5deae860cd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:16:53.053429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1602f-0df9-4270-b4fb-ced73dda7d5f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:53.053774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1602f-0df9-4270-b4fb-ced73dda7d5f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:53.053974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1602f-0df9-4270-b4fb-ced73dda7d5f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:53.055397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 74/149] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:53.061163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:53.061736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:53.062213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:53.074726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2940e-0159-4d16-8ba9-d41b4114f91a None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:53.075016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2940e-0159-4d16-8ba9-d41b4114f91a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:53.075282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2940e-0159-4d16-8ba9-d41b4114f91a None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:53.076895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 75/150] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:53.082305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:53.082591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:53.082867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:53.085272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:53.086795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:53.109858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:53.111079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:53.114580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:53.188271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 76/151] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:53.203212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 76/152] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:53.712426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:53.735136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:53.760006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:53.864838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b1354-5f02-4f9c-ab51-599027ac151a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:53.938499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becdd31-9625-43b3-a7cd-1ae120240a53 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:53.990177 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff72c33-f814-4d1b-ac08-d215b281f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:53.990752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72c33-f814-4d1b-ac08-d215b281f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:53.991032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72c33-f814-4d1b-ac08-d215b281f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:53.991306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72c33-f814-4d1b-ac08-d215b281f0e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.009510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ace5b9-b96f-415a-add1-7cd712e8e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.010165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace5b9-b96f-415a-add1-7cd712e8e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.010442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace5b9-b96f-415a-add1-7cd712e8e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.010717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ace5b9-b96f-415a-add1-7cd712e8e6f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.018681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff72c33-f814-4d1b-ac08-d215b281f0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:54.021946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ace5b9-b96f-415a-add1-7cd712e8e6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9ff72c33-f814-4d1b-ac08-d215b281f0e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:54.023649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:54.024466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:54.025272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 77/153] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-72ace5b9-b96f-415a-add1-7cd712e8e6f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:54.026791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:54.027621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:54.028382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 77/154] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:54.029260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebe004-35db-44ed-a181-2681e245dd0d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.029260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebe004-35db-44ed-a181-2681e245dd0d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.029260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebe004-35db-44ed-a181-2681e245dd0d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.029451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edbd08-ee2b-491e-ad95-7a6fc4da6226 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.029451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edbd08-ee2b-491e-ad95-7a6fc4da6226 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.029569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edbd08-ee2b-491e-ad95-7a6fc4da6226 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.032656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 78/155] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:54.032881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 78/156] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:54.033041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebe004-35db-44ed-a181-2681e245dd0d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:54.037324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f4710-c379-44cd-a8df-33b46dcb2b74 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.037611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f4710-c379-44cd-a8df-33b46dcb2b74 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.037909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f4710-c379-44cd-a8df-33b46dcb2b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.060434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-4c3f4710-c379-44cd-a8df-33b46dcb2b74 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:54.061813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4c3f4710-c379-44cd-a8df-33b46dcb2b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:54.085154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:54.134786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 79/157] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:16:54.135852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.136139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.136414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.173748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:54.175000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:54.177708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:54.270165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebe004-35db-44ed-a181-2681e245dd0d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:54.299241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 80/158] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:54.299964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:54.331086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.332232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.332711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.333256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.334134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:54.334635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:54.336447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:54.357838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 79/159] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:54.363585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:54.459095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e1db7-3931-4d58-a438-03d64cdf3eb0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:54.506919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:54.523731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4a16-e2dd-494f-81c1-f09f4cd5b552 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:54.530094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.530723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.530999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.531269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.531950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:54.532250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:54.533742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:54.544266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 81/160] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:54.544859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:54.566393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:54.585423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.586063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.586347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.586607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.591706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:54.593274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:54.595295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 80/161] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:54 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:54.600595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:54.716281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b20bd-4cd2-41c7-b4d2-c616d25e361f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:54.757842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf83a1-97ba-45ef-b9d6-6032589093ca None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:54.773499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.774203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.774426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.774702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] PATH_INFO: `/v3/users/5a8025e93c084f56aeda2462957536b8` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.775915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8025e93c084f56aeda2462957536b8)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:54.777394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:54.779156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 82/162] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:54 2022] GET /identity/v3/users/5a8025e93c084f56aeda2462957536b8 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:54.785452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:54.815012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.816599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.817181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.817672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:54.847897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:54.881704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:54.930556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08128ae-15f7-42c1-bedb-54b29fcdd528 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:54.950444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:54.952340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:54.986425 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:54.987036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:54.987311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:54.987579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:55.101960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:55.102138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:55.157450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 81/163] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:16:55.158658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:55.214696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:55.291415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 83/164] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:16:55.292159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:55.294428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:55.408210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f5024-6805-490a-94c1-8558180aae31 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:55.449262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:55.449893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:55.450169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:55.450440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:55.455580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd8bf1-4c06-411a-90b5-fa0339a0f209 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:55.462531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:55.467220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:55.468074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:55.468886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 84/165] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:55.469714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:55.507959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:55.533894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:55.535158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:55.539173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:55.539676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:55.562328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:55.574247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:55.575141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:16:55.575922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:55.581850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 82/166] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:55.591701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:55.658113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed8ea-86b0-4d3c-ab14-370c42e41f15 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:55.730278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:55.732108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:55.733717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:55.734207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] PATH_INFO: `/v3/projects/050ae3d43d0140c78af4692911503956` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:55.738860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] RBAC: Authorizing `identity:get_project(project_id=050ae3d43d0140c78af4692911503956)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:55.740646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:55.745510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 85/167] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/projects/050ae3d43d0140c78af4692911503956 => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:55.748064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:55.810970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba5f41-f55e-41c8-871f-64985cc0d0d1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:55.831123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:55.880244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:55.888273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ce7c9-d8af-4087-8de9-eeab0fc64088 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:55.913518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:55.914181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:55.914461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:55.914735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:55.921473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:55.923452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:55.925139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 83/168] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:55.926486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:55.954057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:55.954687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:55.954968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:55.955242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:55.955999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:55.956221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:55.958313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:55.982794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 86/169] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:55.983584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:56.069065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99039b-ffc6-4e48-8146-c5fc0915c02f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.106225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.106833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.107109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.107380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.108045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:56.108343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:56.109755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:56.119115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 84/170] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:56.120210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:56.168501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:56.176357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99c61a-396a-4258-96c6-c6f2f2498757 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.215182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.215796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.216077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.216352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.217121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=050ae3d43d0140c78af4692911503956, user.id=043ada6c6c4a47fdac79dc17270a1821) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:56.217428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:56.218999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:56.231480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc79c7-3f8d-4435-80a6-244505000823 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.233005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 87/171] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:55 2022] GET /identity/v3/role_assignments?user.id=043ada6c6c4a47fdac79dc17270a1821&scope.project.id=050ae3d43d0140c78af4692911503956&role.id=fed56dd461254bc682cf1955076ea721 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:56.234054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:56.271760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.272481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:56.274343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.275040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.275435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.277109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=5a8025e93c084f56aeda2462957536b8) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:56.277521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:56.280135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709d0d1-e483-4d51-8d92-6248a76998bf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:56.290189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 85/172] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:56 2022] GET /identity/v3/role_assignments?user.id=5a8025e93c084f56aeda2462957536b8&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:56.291519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb38ac-286c-451b-9a95-967a2cb33a24 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.291808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb38ac-286c-451b-9a95-967a2cb33a24 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.292284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb38ac-286c-451b-9a95-967a2cb33a24 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.301118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 86/173] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:56.301118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06679d-b8da-4322-837b-d686f4ec9234 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.301118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06679d-b8da-4322-837b-d686f4ec9234 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.301118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06679d-b8da-4322-837b-d686f4ec9234 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.301118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 87/174] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:56.301118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06679d-b8da-4322-837b-d686f4ec9234 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:56.453976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4e424-77c9-4db2-8e58-eb91466038ac None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.491096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06679d-b8da-4322-837b-d686f4ec9234 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.493209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:56.502587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f474ba-ec48-4407-9263-447d825ba238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.503477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f474ba-ec48-4407-9263-447d825ba238 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.504440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f474ba-ec48-4407-9263-447d825ba238 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.505510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f474ba-ec48-4407-9263-447d825ba238 None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.506845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f474ba-ec48-4407-9263-447d825ba238 None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:56.508475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f474ba-ec48-4407-9263-447d825ba238 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:56.510580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 88/175] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:56 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:56.511870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60902ad-474f-4d62-924f-98bac3872571 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.512357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60902ad-474f-4d62-924f-98bac3872571 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.512870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60902ad-474f-4d62-924f-98bac3872571 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.519116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 89/176] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:56.521889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a49545-15e2-4e32-b105-f6e6c4880066 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.522402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a49545-15e2-4e32-b105-f6e6c4880066 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.522873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a49545-15e2-4e32-b105-f6e6c4880066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.545017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b9a49545-15e2-4e32-b105-f6e6c4880066 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:56.546207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b9a49545-15e2-4e32-b105-f6e6c4880066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:56.550468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad63c4-0759-4eed-abb4-5b0cd0a264a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.551381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad63c4-0759-4eed-abb4-5b0cd0a264a0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.552391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad63c4-0759-4eed-abb4-5b0cd0a264a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.553389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad63c4-0759-4eed-abb4-5b0cd0a264a0 None admin] PATH_INFO: `/v3/users/043ada6c6c4a47fdac79dc17270a1821` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.553986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad63c4-0759-4eed-abb4-5b0cd0a264a0 None admin] RBAC: Authorizing `identity:get_user(user_id=043ada6c6c4a47fdac79dc17270a1821)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:56.555502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad63c4-0759-4eed-abb4-5b0cd0a264a0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:56.558039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 88/177] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:56 2022] GET /identity/v3/users/043ada6c6c4a47fdac79dc17270a1821 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:56.559297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.559580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.559853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.581593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:56.583453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:56.639928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 90/178] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:56.641346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.641630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.641903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.659097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:56.661035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:56.662006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 89/179] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:56.662773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:56.683411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:56.757614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 91/180] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:56.758842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:56.777735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:56.781099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a844e2e-bcb4-4920-b0c0-90f93758c06e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.868621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8a438-659b-4960-9343-bda9f548ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.869590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8a438-659b-4960-9343-bda9f548ddeb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.870046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8a438-659b-4960-9343-bda9f548ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.870491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8a438-659b-4960-9343-bda9f548ddeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.883257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbff6b-6f78-4b3e-b4c3-446e568c808d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:56.922466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:56.923567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:56.924101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:56.924633 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:56.947710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:56.995240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8a438-659b-4960-9343-bda9f548ddeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:56.996722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8a438-659b-4960-9343-bda9f548ddeb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.010548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.011920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.099173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.103297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 90/181] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:57.104634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569be2-b103-4e58-97c7-a5dcae539390 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.104960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569be2-b103-4e58-97c7-a5dcae539390 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.105672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569be2-b103-4e58-97c7-a5dcae539390 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.106957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 91/182] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:16:57.109809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569be2-b103-4e58-97c7-a5dcae539390 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.115999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 92/183] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:57.116847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.165803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.213731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569be2-b103-4e58-97c7-a5dcae539390 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.216958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99974a52-2236-4270-8392-462cd5ea1503 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.247318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.247945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.248224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.248494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.250793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.251404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.251678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.251945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.259319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:57.262735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:57.264274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:57.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 92/184] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:57.266258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:57.267353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:57.268157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:16:57.268984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 93/185] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:57.269602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.332635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.374418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40af4c1-5462-42a4-8102-d262be282c73 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.374747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8fbf-97e3-4187-a6d4-6a83605596b7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.385345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.408114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab199c6-1188-4be7-9d25-b11523352238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.408306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.408734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab199c6-1188-4be7-9d25-b11523352238 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.408930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.409023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab199c6-1188-4be7-9d25-b11523352238 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.409202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.409296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab199c6-1188-4be7-9d25-b11523352238 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.409467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.410908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab199c6-1188-4be7-9d25-b11523352238 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.412377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab199c6-1188-4be7-9d25-b11523352238 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.414029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 94/186] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:57.415130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.415407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.415677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.423744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:16:57.429403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:16:57.430411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:16:57.431182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 93/187] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:16:57.431856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.434270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:16:57.435464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:16:57.501511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 95/188] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:16:57.502343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.537176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17acc26-1f95-424c-8061-25ecb0586e0b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.560432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.578150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.578892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.579167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.579438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.580671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.582166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.585234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 94/189] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:57.590686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.603693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633342d-5279-4810-a12c-9377d5f62f0f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.614178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.641863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.642468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.642742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.643022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.643684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:57.643990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.646470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.655019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 96/190] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:57.655805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.704110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c98ce-55e2-41f7-8a32-bce812dbd42c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.741868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.742774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.743246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.743713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.744589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:57.745124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.746690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.756263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 95/191] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:57.761945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.766097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1ab5-49f6-4245-a336-c57a846c86b8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.799364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.799884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.800156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.800422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.801766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] RBAC: Adding query filter params (role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=043ada6c6c4a47fdac79dc17270a1821) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:57.801999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.803509 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.809421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 97/192] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/role_assignments?user.id=043ada6c6c4a47fdac79dc17270a1821&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:57.810190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.814404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.858592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:57.876476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb564ff8-a2b3-486c-8b14-140e2d515061 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.919815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.920442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.920796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.921082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.921769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:57.922191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.923789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:57.935157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 96/193] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:57.936020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:57.947753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f23c7-1043-4d57-a9d8-bbbc141ad5c3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:57.993835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:57.994445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:57.994724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:57.995000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:57.995756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=043ada6c6c4a47fdac79dc17270a1821) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:57.996067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:57.997552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.007526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 98/194] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/role_assignments?user.id=043ada6c6c4a47fdac79dc17270a1821&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=fed56dd461254bc682cf1955076ea721 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:58.008819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.066485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d489a9b-4e81-441f-8dce-92b09e0864ae None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.084647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.111034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.111782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.112064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.112339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] PATH_INFO: `/v3/users/b2e9e197a3ea41f793872ea3ac901584` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.128603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] RBAC: Authorizing `identity:get_user(user_id=b2e9e197a3ea41f793872ea3ac901584)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.130928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.132593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 97/195] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:57 2022] GET /identity/v3/users/b2e9e197a3ea41f793872ea3ac901584 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:58.133429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.141579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.147112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1c5fd-00a0-4169-8ebd-02e4cef2b91a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.196009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.197136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.197591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.198039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] PATH_INFO: `/v3/users/51db1f1a5cae4aeeba8362c7411fdbc0` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.199218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] RBAC: Authorizing `identity:get_user(user_id=51db1f1a5cae4aeeba8362c7411fdbc0)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.202271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.202406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 99/196] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/users/51db1f1a5cae4aeeba8362c7411fdbc0 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:58.203173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.251409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba024a-0622-43ee-bf57-f5df7c8be114 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.286887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.287510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.287777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.288040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.289159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.290671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.292585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 98/197] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:58.293452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.322111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dd0e9-1be7-42ff-a8a4-f4f04180d483 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.336912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.359910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.360539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.360842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.361113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] PATH_INFO: `/v3/users/b2e9e197a3ea41f793872ea3ac901584` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.362248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] RBAC: Authorizing `identity:get_user(user_id=b2e9e197a3ea41f793872ea3ac901584)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.363743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.366225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 100/198] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/users/b2e9e197a3ea41f793872ea3ac901584 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:58.366994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.401565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18cc32-5b2e-4750-86e8-3fd1b0257e19 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.403906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.437940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.438535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.438808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.439076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.470038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1685b2-a3b5-47fa-8921-6c1aac1f4d99 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.504282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.505576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.505692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.505875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.506147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.507046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.545198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.573920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.575333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.607226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 99/199] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:58.608119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.613237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.688742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 101/200] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:58.689891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.718392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbaa215-0e1d-44c7-bd69-e288d9697b5d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.751622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.752217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.752492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.752759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.753450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.754923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.764812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.777327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 100/201] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:16:58 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:16:58.778125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:58.800844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8614c-d991-469c-a2ae-0c91177e8c70 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.841904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.842443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.842724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.842998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.846008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:58.906341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad5c9f-2726-4eba-abcb-2bc49747c2ac None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:58.925395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.926993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.948714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:58.949372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:58.949648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:58.949916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:58.951062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:58.952512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:58.956836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 101/202] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:58.956836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:59.037727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:59.068584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 102/203] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:16:59.069506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:59.095493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26380f52-f63b-419d-b845-043f1a799353 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:59.113922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:59.143190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:59.144184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:59.144562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:59.144972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:59.146284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:59.148621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:59.154134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 102/204] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:58 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:59.154134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:59.201132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53307ac-8249-44c9-8f4b-86d0cfc46d76 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:59.242918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:59.243582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:59.243860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:59.244128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:59.245337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=b2e9e197a3ea41f793872ea3ac901584) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:59.245659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:59.247561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:59.256084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 103/205] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:59 2022] GET /identity/v3/role_assignments?user.id=b2e9e197a3ea41f793872ea3ac901584&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:59.257050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:59.275149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0cde-de88-4088-b527-17d74d03386d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:59.310784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:59.319264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:59.319955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:59.320232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:59.320500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:59.321880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:59.323627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:59.325693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 103/206] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:16:59 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:59.331338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:16:59.394735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:16:59.407409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a763f6-02ba-4657-af77-e66993777ede None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:59.465513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a188346-92a6-4e23-81d6-8840675cadb4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:16:59.493816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:59.494443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:59.494722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:59.494999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:59.496054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] RBAC: Adding query filter params (role.id=ed708940f2d64bfab650ce5deae860cd, scope.project.id=1b7cc325ca0144afbe3094beaaf32144, user.id=51db1f1a5cae4aeeba8362c7411fdbc0) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:59.496054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:59.497693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8272b-83e5-4dd1-804c-6862315f8b5c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:59.505854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 104/207] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:59 2022] GET /identity/v3/role_assignments?user.id=51db1f1a5cae4aeeba8362c7411fdbc0&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=ed708940f2d64bfab650ce5deae860cd => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:16:59.547726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:16:59.548706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:16:59.549226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:16:59.549702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:16:59.550645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] RBAC: Adding query filter params (role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=b2e9e197a3ea41f793872ea3ac901584) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:16:59.551147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:16:59.553230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb9058-80f7-4ab0-a2d8-734ea5a32915 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:16:59.561331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 104/208] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:16:59 2022] GET /identity/v3/role_assignments?user.id=b2e9e197a3ea41f793872ea3ac901584&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:00.893074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd7ba4-bb7f-4db8-bf20-61b66d27816a None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:00.893602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd7ba4-bb7f-4db8-bf20-61b66d27816a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:00.893602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd7ba4-bb7f-4db8-bf20-61b66d27816a None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:00.894973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 105/209] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:00.900600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec83d5-78d6-4edd-b317-0259a0e87d3a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:00.901106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec83d5-78d6-4edd-b317-0259a0e87d3a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:00.901184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec83d5-78d6-4edd-b317-0259a0e87d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:00.920247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8aec83d5-78d6-4edd-b317-0259a0e87d3a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:00.921489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8aec83d5-78d6-4edd-b317-0259a0e87d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:00.983382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:00.994480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 105/210] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:01.123886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e27b6-7d03-4302-a214-66bf941cb883 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.124204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e27b6-7d03-4302-a214-66bf941cb883 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.124437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e27b6-7d03-4302-a214-66bf941cb883 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.125842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 106/211] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:01.131453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d62aa-e0be-4a6a-899d-63cb70451fad None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.131737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d62aa-e0be-4a6a-899d-63cb70451fad None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.132015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d62aa-e0be-4a6a-899d-63cb70451fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.150443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6b1d62aa-e0be-4a6a-899d-63cb70451fad None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:01.151565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6b1d62aa-e0be-4a6a-899d-63cb70451fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:01.223017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 106/212] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:01.271013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e27b6-7d03-4302-a214-66bf941cb883 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:01.383778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e27b6-7d03-4302-a214-66bf941cb883 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:01.424174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:01.424847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.425128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.425405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.437083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:01.442312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:01.443253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:01.446077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 107/213] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:01.450414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abfb60-93b5-410e-8ad0-7e2d4d1d22d5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.450414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abfb60-93b5-410e-8ad0-7e2d4d1d22d5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.450414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abfb60-93b5-410e-8ad0-7e2d4d1d22d5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.452069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:01.452288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 107/214] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:01.457061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.459432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.459432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.479121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:01.479784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:01.480707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:01.569726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 108/215] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:01.576014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48057dfb-da35-4c96-86a3-74da948f2d1f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:01.577927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:01.612006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:01.612626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.612932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.613262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.613869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:01.614173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:01.615581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53f82-00ef-49b6-9c2b-7978101a8bb6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:01.624803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 108/216] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:01.625667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ba6f0-4cdc-4137-8fe1-5ed06c621529 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.625947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ba6f0-4cdc-4137-8fe1-5ed06c621529 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.626220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ba6f0-4cdc-4137-8fe1-5ed06c621529 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.627548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 109/217] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:01.630287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ba6f0-4cdc-4137-8fe1-5ed06c621529 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:01.652949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:01.705125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5670ea-8cb3-4e56-8432-260a3602458b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:01.723593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:01.742227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ba6f0-4cdc-4137-8fe1-5ed06c621529 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:01.742571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5a3d4-37a2-4fa6-969f-2cd7af9ce9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:01.743205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5a3d4-37a2-4fa6-969f-2cd7af9ce9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.743484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5a3d4-37a2-4fa6-969f-2cd7af9ce9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.743769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5a3d4-37a2-4fa6-969f-2cd7af9ce9d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.745161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5a3d4-37a2-4fa6-969f-2cd7af9ce9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:01.746353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5a3d4-37a2-4fa6-969f-2cd7af9ce9d3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:01.747929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 109/218] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:01.749056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.749337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.749609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.768523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:01.769750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:01.784487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:01.785288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.785410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.785671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] PATH_INFO: `/v3/users/e087cbd5013f46deb618c65368a36c75` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.800025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] RBAC: Authorizing `identity:get_user(user_id=e087cbd5013f46deb618c65368a36c75)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:01.801577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:01.808360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 110/219] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:01 2022] GET /identity/v3/users/e087cbd5013f46deb618c65368a36c75 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:01.808360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:01.853113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 110/220] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:01.861046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:01.904911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:01.924075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e694-2e90-488e-acfb-aca05768a6a5 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:01.962602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb99d47-dd71-4ce9-a334-b512f74a6054 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:01.962761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:01.963300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:01.963577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:01.963855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:01.964580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:01.965988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:01.984845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:01.995721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 111/221] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:01 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:01.996532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.004883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.005877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.006323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.006764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.110629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.112438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.134884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c41ab5-24ed-4414-8b06-61f38bc422e6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.175299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:02.183501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.184352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.184635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.184943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.196397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:02.201595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:02.202461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:02.203228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 112/222] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:02.203865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.244932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 111/223] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:02.246659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.286586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:02.337573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b568a6-466f-483f-b6cd-98e4fca135e1 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.374946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9b396-5951-483f-b286-f55bab54b5e3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.381713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.382576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.382997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.383965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.399406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:02.407438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:02.408512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:02.409530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:02.412961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 113/224] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:02.414153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.416190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.416942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.417594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.418068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.419107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:02.419478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.420967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.431942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 112/225] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.432866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.449601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:02.542598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c30c8-623c-494c-8752-1c375f44bbcf None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.552040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0825c3e-5793-470e-a1e0-7e6ab073c53c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.562506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:02.583504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.584792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.585350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.585756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.586896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.588348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.589475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.590022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 114/226] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.590402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.590687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.590783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.590949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.591643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:02.591942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.593453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.603915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 113/227] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.604941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.689340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:02.717693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac85ec-45ef-480a-8138-6a783213c7d0 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.718972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b2e9-efa9-43cd-bcdb-a68b5f6a3ee9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.753393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.754117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.754399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.754670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] PATH_INFO: `/v3/users/b2e9e197a3ea41f793872ea3ac901584` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.755816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] RBAC: Authorizing `identity:get_user(user_id=b2e9e197a3ea41f793872ea3ac901584)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.756908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.757350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.758105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.758604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.763796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.763939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 115/228] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/users/b2e9e197a3ea41f793872ea3ac901584 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.763939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.765195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=e087cbd5013f46deb618c65368a36c75) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:02.765725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.767381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.774462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 114/229] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/role_assignments?user.id=e087cbd5013f46deb618c65368a36c75&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=fed56dd461254bc682cf1955076ea721 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.775632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.829849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:02.895190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36bbdf-08e9-48cc-a116-632439199dc3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.915519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8fa9b-79c3-4d64-a44f-f2298f0c42e9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:02.937600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.938411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.938697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.938975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] PATH_INFO: `/v3/users/b2e9e197a3ea41f793872ea3ac901584` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.945723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] RBAC: Authorizing `identity:get_user(user_id=b2e9e197a3ea41f793872ea3ac901584)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.947249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.953435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 116/230] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/users/b2e9e197a3ea41f793872ea3ac901584 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.957081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:02.958401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:02.958882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:02.958982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:02.959974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:02.961628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:02.963384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:02.965400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 115/231] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:02.984684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:03.006785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:03.109443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8836-5655-4b3c-a6c2-fe4444493d52 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:03.148716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:03.149536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:03.149810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:03.150076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77/users/b2e9e197a3ea41f793872ea3ac901584/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:03.152434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] RBAC: Authorizing `identity:create_grant(project_id=0d1859c09d264db597aab109f2d67e77, user_id=b2e9e197a3ea41f793872ea3ac901584, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:03.154430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:03.173913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 116/232] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:02 2022] PUT /identity/v3/projects/0d1859c09d264db597aab109f2d67e77/users/b2e9e197a3ea41f793872ea3ac901584/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:17:03.188370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c163019-1de8-4d43-97a5-c1e3b867362d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:03.215135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:03.229047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9253027-79e7-45e4-91d7-111035eaeb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:03.229686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9253027-79e7-45e4-91d7-111035eaeb78 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:03.229962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9253027-79e7-45e4-91d7-111035eaeb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:03.230430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9253027-79e7-45e4-91d7-111035eaeb78 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:03.231621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9253027-79e7-45e4-91d7-111035eaeb78 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:03.233106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9253027-79e7-45e4-91d7-111035eaeb78 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:03.235213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 117/233] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:02 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:03.242116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:03.320753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:03.361124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a6b09-c0e1-47ea-8aed-200e3c5068dc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:03.417578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a48a8c-5e17-4549-9ec6-6749a4b85f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:03.418282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48a8c-5e17-4549-9ec6-6749a4b85f93 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:03.418561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48a8c-5e17-4549-9ec6-6749a4b85f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:03.418838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48a8c-5e17-4549-9ec6-6749a4b85f93 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77/users/b2e9e197a3ea41f793872ea3ac901584/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:03.421327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a48a8c-5e17-4549-9ec6-6749a4b85f93 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d1859c09d264db597aab109f2d67e77, user_id=b2e9e197a3ea41f793872ea3ac901584, role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:03.422895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a48a8c-5e17-4549-9ec6-6749a4b85f93 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:03.456819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 117/234] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:03 2022] PUT /identity/v3/projects/0d1859c09d264db597aab109f2d67e77/users/b2e9e197a3ea41f793872ea3ac901584/roles/7eef59c95b8945768aab3374bc2721f1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:04.167076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c5224-bddd-4bf6-a234-fbc71178dece None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:04.167552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c5224-bddd-4bf6-a234-fbc71178dece None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:04.167552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c5224-bddd-4bf6-a234-fbc71178dece None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:04.168889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 118/235] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:04.173996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769d59f-b274-4420-8cc1-8020ea083997 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:04.174369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769d59f-b274-4420-8cc1-8020ea083997 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:04.174544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769d59f-b274-4420-8cc1-8020ea083997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:04.201771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2769d59f-b274-4420-8cc1-8020ea083997 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:04.202976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2769d59f-b274-4420-8cc1-8020ea083997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:04.286140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 118/236] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:04.556398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d548d1b-32b6-45f5-9ba7-fe4066c8d502 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:04.556733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d548d1b-32b6-45f5-9ba7-fe4066c8d502 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:04.556934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d548d1b-32b6-45f5-9ba7-fe4066c8d502 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:04.558513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 119/237] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:04.563493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:04.563778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:04.564050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:04.589828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:04.591044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:04.678541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:04.697035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 119/238] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:04.698404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d548d1b-32b6-45f5-9ba7-fe4066c8d502 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:04.807249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d548d1b-32b6-45f5-9ba7-fe4066c8d502 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:04.843636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:04.844647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:04.845157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:04.845621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:04.858789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:04.864701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:04.865637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:04.866432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:04.872946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 120/239] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:04.881930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.007288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366ac20-54e0-4a1d-bf2a-4ac4f1b36040 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:05.048106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:05.048728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.049031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.049303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.049971 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:05.050791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:05.052179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:05.060954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 120/240] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:05.067363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.104248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.161283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:05.161865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:05.252356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1584c9e-17e5-4dd9-9360-a2eb4f89cbe9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:05.270862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56852-b16c-4cc9-849e-0b47f106bb22 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:05.293965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:05.295059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.295399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.295873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] PATH_INFO: `/v3/users/abe58e23a61e4e049782593e95002d48` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.308243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341737ab-53f0-412b-b4e2-2a85e2f67640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:05.308912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341737ab-53f0-412b-b4e2-2a85e2f67640 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.309188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341737ab-53f0-412b-b4e2-2a85e2f67640 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.309453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341737ab-53f0-412b-b4e2-2a85e2f67640 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.310698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341737ab-53f0-412b-b4e2-2a85e2f67640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:05.310874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] RBAC: Authorizing `identity:get_user(user_id=abe58e23a61e4e049782593e95002d48)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:05.312211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341737ab-53f0-412b-b4e2-2a85e2f67640 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:05.312319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:05.314081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 121/241] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:05.314309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 121/242] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:05 2022] GET /identity/v3/users/abe58e23a61e4e049782593e95002d48 => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:05.315870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9d110-8513-46bc-b606-0af7181a7c9b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.316890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9d110-8513-46bc-b606-0af7181a7c9b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.317175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9d110-8513-46bc-b606-0af7181a7c9b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.318632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 122/243] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:05.320543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.323507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9d110-8513-46bc-b606-0af7181a7c9b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.426809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb61b-090f-49eb-b98e-9c945516eeae None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:05.445236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:05.455303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9d110-8513-46bc-b606-0af7181a7c9b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:05.465258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97e4e-59a1-4674-8bb8-ced0de075fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:05.466181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97e4e-59a1-4674-8bb8-ced0de075fa5 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.466651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97e4e-59a1-4674-8bb8-ced0de075fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.467110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97e4e-59a1-4674-8bb8-ced0de075fa5 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.467963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e97e4e-59a1-4674-8bb8-ced0de075fa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:05.469675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e97e4e-59a1-4674-8bb8-ced0de075fa5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:05.482271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:05.495368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 122/244] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:05 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:05.497130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.497634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.498211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.504225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8f014-d685-4f77-ba3c-49cc100af290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:05.504896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8f014-d685-4f77-ba3c-49cc100af290 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.505178 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8f014-d685-4f77-ba3c-49cc100af290 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.505450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8f014-d685-4f77-ba3c-49cc100af290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.521918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:05.523482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:05.586214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8f014-d685-4f77-ba3c-49cc100af290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:05.587789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8f014-d685-4f77-ba3c-49cc100af290 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:05.600152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 123/245] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:05.720655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 123/246] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:05.728278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.843065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:05.850198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e3726-09ff-4d8e-baa9-4ab65246f449 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:05.867114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e11d84-46b1-40e5-88ac-9729fb049d2c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.867558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e11d84-46b1-40e5-88ac-9729fb049d2c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.867929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e11d84-46b1-40e5-88ac-9729fb049d2c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.869713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 124/247] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:05.875615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b792b8-93ea-4299-b130-0ad56096471b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.876056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b792b8-93ea-4299-b130-0ad56096471b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.876465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b792b8-93ea-4299-b130-0ad56096471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.897328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:05.897959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:05.898236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:05.898501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:05.899760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:05.901322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:05.907845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 124/248] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:05 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:05.909470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:05.911506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-26b792b8-93ea-4299-b130-0ad56096471b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:05.914254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-26b792b8-93ea-4299-b130-0ad56096471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:05.928504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:06.004230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 125/249] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:06.005976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f428097-a5e8-4a05-9435-7293c9fe1c92 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.006402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f428097-a5e8-4a05-9435-7293c9fe1c92 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.006830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f428097-a5e8-4a05-9435-7293c9fe1c92 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.008841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 126/250] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:06.009985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f428097-a5e8-4a05-9435-7293c9fe1c92 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.050947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa660-1ab9-4b83-b014-9f79f522ba4e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:06.099658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:06.100567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.101035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.101448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.102333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] RBAC: Adding query filter params (role.id=ed708940f2d64bfab650ce5deae860cd, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=abe58e23a61e4e049782593e95002d48) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:06.102781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:06.104392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680444c-ea05-48c7-98df-66c24c1d287b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:06.114174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 125/251] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:05 2022] GET /identity/v3/role_assignments?user.id=abe58e23a61e4e049782593e95002d48&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=ed708940f2d64bfab650ce5deae860cd => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:06.115573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.116010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.117326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.149748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:06.152668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:06.165588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f428097-a5e8-4a05-9435-7293c9fe1c92 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:06.201393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:06.222433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:06.223513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.223960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.224376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.244337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:06.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:06.254475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:06.255434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:06.262810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 127/252] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:06.268985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.301812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:06.336286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 126/253] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:06.337340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.425501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d142264-ab60-40c8-b599-677128c13630 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:06.478944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:06.479607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.479884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.480162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.480906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:06.481215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:06.482611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:06.511688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 128/254] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:06.522235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.547541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb313e5-1ac3-481f-a55f-c9c71fff3dca None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:06.613489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:06.614132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.614409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.614677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.630646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:06.644358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:06.644724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:06.645592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:06.646403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 127/255] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:06.653237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.671967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461f2b6-9817-4494-8897-e85047584a76 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:06.706056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:06.722133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:06.723153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.723636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.724138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] PATH_INFO: `/v3/users/e087cbd5013f46deb618c65368a36c75` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.729611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] RBAC: Authorizing `identity:get_user(user_id=e087cbd5013f46deb618c65368a36c75)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:06.731507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:06.734142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 129/256] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/users/e087cbd5013f46deb618c65368a36c75 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:06.735332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.837485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124782a6-f57b-467c-a036-8a163e6dbbef None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:06.885324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:06.886571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:06.887071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:06.887560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:06.888960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:06.889470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:06.891080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:06.903845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 128/257] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:06.911412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:06.947797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26779d-ea1b-493d-9e92-96720d0c510c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:07.014439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:07.014988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:07.015261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:07.015529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:07.022317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:07.035573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:07.040820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:07.041689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:07.042473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 130/258] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:07.043124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:07.074314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:07.138697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47a25-5af2-4b64-b8df-e23cc3207d53 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:07.182086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abbff-8413-45fa-8cb7-57b46001b102 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:07.218124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:07.218537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:07.218799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:07.219611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] PATH_INFO: `/v3/users/b2e9e197a3ea41f793872ea3ac901584` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:07.220838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=b2e9e197a3ea41f793872ea3ac901584)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:07.222261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:07.224277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 131/259] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:07 2022] GET /identity/v3/users/b2e9e197a3ea41f793872ea3ac901584 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:07.225069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:07.226923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:07.227562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:07.227840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:07.228112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:07.230315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:07.231756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:07.233424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 129/260] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:06 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:07.238804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:07.331805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:07.401888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddbf5c-e5fb-418e-9527-3d7214785ed9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:07.444028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeae5b0-384b-4985-bc33-aa04af617176 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:07.477988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:07.478604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:07.478885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:07.481253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:07.481937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:07.482245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:07.483653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:07.506130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:07.506737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:07.507215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 132/261] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:07.507319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:07.507319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:07.507816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:07.536793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:07.588246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:07.589898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:07.704220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 130/262] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:07.709374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:07.793652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ee219-a49d-494a-b7db-61087e05f11f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:07.813114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:07.842537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:07.864093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:07.864774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:07.865055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:07.865327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77/users/e087cbd5013f46deb618c65368a36c75/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:07.867341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] RBAC: Authorizing `identity:create_grant(project_id=0d1859c09d264db597aab109f2d67e77, user_id=e087cbd5013f46deb618c65368a36c75, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:07.868818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:07.891591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 133/263] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:07 2022] PUT /identity/v3/projects/0d1859c09d264db597aab109f2d67e77/users/e087cbd5013f46deb618c65368a36c75/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:07.892547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:07.942073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0be87-d9df-4158-93b4-49a47c89beb6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:08.030308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:08.031161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.031482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.031754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] PATH_INFO: `/v3/users/b2e9e197a3ea41f793872ea3ac901584` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.037671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] RBAC: Authorizing `identity:get_user(user_id=b2e9e197a3ea41f793872ea3ac901584)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:08.039182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:08.040899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 131/264] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:07 2022] GET /identity/v3/users/b2e9e197a3ea41f793872ea3ac901584 => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:08.049063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:08.175075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d03b-86d7-44a3-b4dc-2a38a30da2bd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:08.179676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a20e92-57f1-4aa1-a7b4-61d3bee29e11 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:08.197787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:08.226432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:08.227291 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.227564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.227828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.281191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:08.281875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.282185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.282464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.285488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:08.286982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:08.288797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 134/265] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:07 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:08.306199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:08.313662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:08.315157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:08.416217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:08.507070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aae64-f703-4bf9-a9fa-b375999e2573 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:08.509271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 132/266] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:08.515673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:08.542361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:08.578643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:08.579373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.579732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.580080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.580843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] RBAC: Adding query filter params (role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=b2e9e197a3ea41f793872ea3ac901584) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:08.581280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:08.582728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57baee-8b80-4565-a0e8-1b77a2b317e7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:08.588806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 135/267] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:08 2022] GET /identity/v3/role_assignments?user.id=b2e9e197a3ea41f793872ea3ac901584&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:08.590047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84ae6c-960a-4a99-944c-0ec1ca4bdbfa None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.590327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84ae6c-960a-4a99-944c-0ec1ca4bdbfa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.590599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84ae6c-960a-4a99-944c-0ec1ca4bdbfa None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.591878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 136/268] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:08.593759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412afd4-7bd5-4596-aad9-676efcbcb065 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.594144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412afd4-7bd5-4596-aad9-676efcbcb065 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.594425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412afd4-7bd5-4596-aad9-676efcbcb065 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.600222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 137/269] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:08.601700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e2406-d9b4-44a5-b9ac-7e883324813a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.601983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e2406-d9b4-44a5-b9ac-7e883324813a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.602260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e2406-d9b4-44a5-b9ac-7e883324813a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.603657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 138/270] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:08.610107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526d0b-e336-49d5-bc7e-82f693a2775a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.610391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526d0b-e336-49d5-bc7e-82f693a2775a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.610670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526d0b-e336-49d5-bc7e-82f693a2775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.630531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a4526d0b-e336-49d5-bc7e-82f693a2775a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:08.631766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a4526d0b-e336-49d5-bc7e-82f693a2775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:08.642156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af933a-1dd2-4ba2-80eb-a319044d50ad None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:08.686211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc4047-544f-49ba-ae18-d39a4fd659e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:08.686853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc4047-544f-49ba-ae18-d39a4fd659e8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.687197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc4047-544f-49ba-ae18-d39a4fd659e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.687445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc4047-544f-49ba-ae18-d39a4fd659e8 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.688641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc4047-544f-49ba-ae18-d39a4fd659e8 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:08.690262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc4047-544f-49ba-ae18-d39a4fd659e8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:08.692507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 133/271] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:08 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:08.693842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.694126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.694400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.715015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:08.715689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 139/272] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:08.716309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:08.717688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:08.718162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:08.718611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:08.738949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:08.740551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:08.808872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:08.809499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 134/273] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:08.810640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:08.842654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:08.849386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 140/274] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:08.986318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae4d56-f3ff-4e6d-85c9-daf83f490c18 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.030299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.030903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.031217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.031452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.032186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=b2e9e197a3ea41f793872ea3ac901584) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:09.032492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.033975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.041501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 135/275] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:08 2022] GET /identity/v3/role_assignments?user.id=b2e9e197a3ea41f793872ea3ac901584&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:09.044698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.156852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.169430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8ad73-f2ad-4cbf-bf4e-bbbdd9cc6549 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.206855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.207484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.207759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.208030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.221279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:09.225970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:09.226841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:09.227609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 141/276] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:09.233428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.284647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:09.297073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:09.317477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281451-c238-4e2f-8754-51b1bf86024a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.359148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.359782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.361549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.361549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.378136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:09.378810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6b8f5-c2f7-4a78-a10b-eaaa03a91922 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:09.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:09.386373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:09.387365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:09.389705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 136/277] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:09.390641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.423147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.423766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.424111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.424450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.425318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:09.425712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.427246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.436996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 142/278] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:09.442417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.510321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ef629-c48b-42f0-a535-57fc4fd3f535 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.548889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:09.553461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318231b-6130-4e44-9d62-0acd35c6ee4b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.555906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.556691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.557136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.557545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.558781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.560580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.562639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 137/279] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:09.563541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.569743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:09.604645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.605294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.605573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.605848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.606592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:09.606919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.608447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.620815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 143/280] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:09.620815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.679147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf60d-38f5-4cd3-ac41-4e1a8bead6de None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.717869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.718488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.718762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.719034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] PATH_INFO: `/v3/users/2d0eab97bba841b79b2526391559a157` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.732653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] RBAC: Authorizing `identity:get_user(user_id=2d0eab97bba841b79b2526391559a157)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.734132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.735752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 138/281] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/users/2d0eab97bba841b79b2526391559a157 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:09.736491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.748539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14d015-269b-448c-af4e-8a46b174c598 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.788915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.789507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.789782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.790044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.790672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.792100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.801791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:09.821553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 144/282] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:09 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:09.823477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.855920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:09.872821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33c1e0-7643-4659-86dc-eca365f83db0 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:09.942623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:09.943897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:09.944372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:09.944809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] PATH_INFO: `/v3/users/abe58e23a61e4e049782593e95002d48` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:09.946581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=abe58e23a61e4e049782593e95002d48)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:09.951278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:09.960946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 139/283] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/users/abe58e23a61e4e049782593e95002d48 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:09.966743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:09.990337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416a408-4498-4c59-926d-b08a650ef7c5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:10.031383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:10.032360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:10.032639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:10.033070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:10.033888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, user.id=2d0eab97bba841b79b2526391559a157, scope.system=all) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:10.034263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:10.035805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:10.042096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 145/284] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/role_assignments?user.id=2d0eab97bba841b79b2526391559a157&scope.system=all&role.id=fed56dd461254bc682cf1955076ea721 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:10.140274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809839ff-1b89-43ce-b10a-3adaf9166bc6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:10.175918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:10.183751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230224d6-b9db-40cd-b85e-7cc760989585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:10.184467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230224d6-b9db-40cd-b85e-7cc760989585 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:10.184720 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230224d6-b9db-40cd-b85e-7cc760989585 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:10.185017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230224d6-b9db-40cd-b85e-7cc760989585 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:10.187742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230224d6-b9db-40cd-b85e-7cc760989585 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:10.192065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230224d6-b9db-40cd-b85e-7cc760989585 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:10.192065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 140/285] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:09 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:10.197725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:10.298733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:10.319262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10130789-34bb-43a0-b104-c6e76d53e74f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:10.362337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aaab4-819f-4cea-8875-1229529497d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:10.363004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aaab4-819f-4cea-8875-1229529497d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:10.363280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aaab4-819f-4cea-8875-1229529497d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:10.363614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aaab4-819f-4cea-8875-1229529497d8 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77/users/abe58e23a61e4e049782593e95002d48/roles/ed708940f2d64bfab650ce5deae860cd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:10.365585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61aaab4-819f-4cea-8875-1229529497d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d1859c09d264db597aab109f2d67e77, user_id=abe58e23a61e4e049782593e95002d48, role_id=ed708940f2d64bfab650ce5deae860cd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:10.367033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61aaab4-819f-4cea-8875-1229529497d8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:10.384312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 146/286] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:10 2022] PUT /identity/v3/projects/0d1859c09d264db597aab109f2d67e77/users/abe58e23a61e4e049782593e95002d48/roles/ed708940f2d64bfab650ce5deae860cd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:10.552576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5e94-595b-49da-9739-44b09b77ec8f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:10.553071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5e94-595b-49da-9739-44b09b77ec8f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:10.553071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5e94-595b-49da-9739-44b09b77ec8f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:10.554412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 141/287] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:10.559933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:10.560367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:10.560473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:10.581921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:10.583257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:10.666589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 147/288] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:11.111402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5e94-595b-49da-9739-44b09b77ec8f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:11.242443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5e94-595b-49da-9739-44b09b77ec8f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:11.299303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:11.299932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:11.300204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:11.300470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:11.314651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:11.319279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:11.320128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:11.320918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 142/289] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:11.326088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:11.445714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:11.484561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846919-4a56-4f08-8deb-000d9cd3210b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:11.538006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:11.538535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:11.538808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:11.539073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:11.539742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:11.540036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:11.541428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:11.552709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 148/290] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:11.558469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:11.674925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:11.733128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed703af2-29a5-4bcf-b3bd-4c800127adcd None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:11.783844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:11.784901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:11.785375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:11.785836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] PATH_INFO: `/v3/users/e087cbd5013f46deb618c65368a36c75` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:11.787193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] RBAC: Authorizing `identity:get_user(user_id=e087cbd5013f46deb618c65368a36c75)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:11.788914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:11.790705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 143/291] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:11 2022] GET /identity/v3/users/e087cbd5013f46deb618c65368a36c75 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:11.796973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:11.936688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073e19c-a087-4151-8d73-d2041f14c743 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:11.984223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6579d47-c07f-432a-806a-ad4d7d9e1d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:11.985255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6579d47-c07f-432a-806a-ad4d7d9e1d8f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:11.985729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6579d47-c07f-432a-806a-ad4d7d9e1d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:11.986193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6579d47-c07f-432a-806a-ad4d7d9e1d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:11.995965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:12.079733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6579d47-c07f-432a-806a-ad4d7d9e1d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:12.081606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6579d47-c07f-432a-806a-ad4d7d9e1d8f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:12.233920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 149/292] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:12.239809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:12.344122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331b685-d282-47d9-b31d-76945cc220a0 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.344440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331b685-d282-47d9-b31d-76945cc220a0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.344604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331b685-d282-47d9-b31d-76945cc220a0 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.346100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 150/293] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:12.359686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b249-5d7a-445f-91b3-b8396e4ae2fd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.359966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b249-5d7a-445f-91b3-b8396e4ae2fd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.360249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b249-5d7a-445f-91b3-b8396e4ae2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.380227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c09e0-49ea-49fe-babf-81cb6eccfa04 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:12.386198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0b85b249-5d7a-445f-91b3-b8396e4ae2fd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:12.387404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0b85b249-5d7a-445f-91b3-b8396e4ae2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:12.442105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:12.457116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:12.458622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:12.459261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.459543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.459816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.461472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:12.463092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:12.473820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 144/294] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:12 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:12.480018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:12.512320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 151/295] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:12.514390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf719b5-1afc-4f3a-aa10-2f0c1c686534 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.514679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf719b5-1afc-4f3a-aa10-2f0c1c686534 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.514955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf719b5-1afc-4f3a-aa10-2f0c1c686534 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.517069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 152/296] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:12.522463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb7413-c459-4051-890e-a2b5b27e338e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.522882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb7413-c459-4051-890e-a2b5b27e338e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.523164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb7413-c459-4051-890e-a2b5b27e338e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.541820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6cfb7413-c459-4051-890e-a2b5b27e338e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:12.543034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6cfb7413-c459-4051-890e-a2b5b27e338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:12.601966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22242359-ad68-47ab-91ff-5cb00623a664 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:12.614315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 153/297] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:12.640398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:12.641121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.641720 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.642170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.643075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=e087cbd5013f46deb618c65368a36c75) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:12.643555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:12.645279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9cde5-3381-46c6-80c3-262d5944dcd3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:12.651261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63954de9-eef3-43f7-bb3f-3f6510874d0c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.651832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63954de9-eef3-43f7-bb3f-3f6510874d0c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.652303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63954de9-eef3-43f7-bb3f-3f6510874d0c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.652621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 145/298] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:12 2022] GET /identity/v3/role_assignments?user.id=e087cbd5013f46deb618c65368a36c75&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=fed56dd461254bc682cf1955076ea721 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:12.654432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 154/299] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:12.660185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8493-9c8d-40ce-b2cd-b1e48d31896c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.660876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8493-9c8d-40ce-b2cd-b1e48d31896c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.661351 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8493-9c8d-40ce-b2cd-b1e48d31896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.693483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f50c8493-9c8d-40ce-b2cd-b1e48d31896c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:12.695045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f50c8493-9c8d-40ce-b2cd-b1e48d31896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:12.784051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:12.803520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 146/300] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:12.902890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63954de9-eef3-43f7-bb3f-3f6510874d0c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:12.980611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312c04-acd3-404e-9dca-95f434e12fa3 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.980949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312c04-acd3-404e-9dca-95f434e12fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.981205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312c04-acd3-404e-9dca-95f434e12fa3 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:12.982559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 147/301] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:12.987824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:12.988106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:12.988380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.010722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.013329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:13.014547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:13.019880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63954de9-eef3-43f7-bb3f-3f6510874d0c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.059052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf74404-760f-49a9-8659-31866764900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.059720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf74404-760f-49a9-8659-31866764900f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.059997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf74404-760f-49a9-8659-31866764900f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.060270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf74404-760f-49a9-8659-31866764900f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.061418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf74404-760f-49a9-8659-31866764900f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.062871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf74404-760f-49a9-8659-31866764900f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.064836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 155/302] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:13.065705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf74404-760f-49a9-8659-31866764900f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.100651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 148/303] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:13.105760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.177094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf74404-760f-49a9-8659-31866764900f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.213716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.214347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.214636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.214891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.220398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937fe52-5db1-4c34-b968-e2884ac28c52 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.225977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:13.231770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:13.233513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 156/304] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:13.234166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.246519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.260893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.268170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.268912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.269198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.269475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] PATH_INFO: `/v3/groups` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.270101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.271545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.289244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 149/305] 10.222.0.81 () {60 vars in 1134 bytes} [Wed Aug 24 03:17:13 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:13.298225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.356277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbcd17-fd49-49f0-a09e-0bf5371e5bcb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.399510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.400063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.400339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.400610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.413445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:13.418962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:13.419868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:13.420735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 157/306] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:13.421411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.442514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeef57-640a-47a4-8700-edb09fb20fe3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.484409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.485050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.485330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.485602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.486276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:13.486577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.487962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.497596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 150/307] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:13.502161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.525999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.546942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af504a11-246d-444c-a5b3-192c5dd71a28 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.547217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.584449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.585511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.585789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.586060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.598466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:13.604942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:13.605945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:13.606731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 158/308] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:13.607429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.617151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb87e2-a42f-4a2c-87a7-0538a9d6299a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.655813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.656418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.656689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.656982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.657643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] RBAC: Adding query filter params (name=member) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:13.657947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.659336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.669933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 151/309] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:13.670732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.725084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17675b0f-15a8-4b5b-9083-812b4f8d3699 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.760738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.761405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.761676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.761940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] PATH_INFO: `/v3/users/babfe08cd0944213b282bba8cf3ca3fc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.774233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.778218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] RBAC: Authorizing `identity:get_user(user_id=babfe08cd0944213b282bba8cf3ca3fc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.780184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.782030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 159/310] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/users/babfe08cd0944213b282bba8cf3ca3fc => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:13.782784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9322b5-58f9-4fee-8c58-99988dad920a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.783164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.787680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:13.826471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.827104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.827378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.827650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.828357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:13.828653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.830323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.839797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 152/311] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:13.840393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.900458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa29b9-0e00-4692-bdf1-87a32c72cc3f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:13.941436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:13.942239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:13.942376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:13.942696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] PATH_INFO: `/v3/users/2d0eab97bba841b79b2526391559a157` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:13.943955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] RBAC: Authorizing `identity:get_user(user_id=2d0eab97bba841b79b2526391559a157)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:13.946314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:13.950283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 160/312] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/users/2d0eab97bba841b79b2526391559a157 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:13.950921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:13.965464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff947f3-33c4-47bd-a182-6cf151509b87 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:14.002488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:14.003027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:14.003281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:14.003555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:14.004283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] RBAC: Adding query filter params (role.id=ed708940f2d64bfab650ce5deae860cd, user.id=babfe08cd0944213b282bba8cf3ca3fc, scope.system=all) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:14.004589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:14.006062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:14.020795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 153/313] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/role_assignments?user.id=babfe08cd0944213b282bba8cf3ca3fc&scope.system=all&role.id=ed708940f2d64bfab650ce5deae860cd => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:14.020795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:14.035222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:14.063825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:14.082871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a5816-df3b-4069-a764-164f6ed26480 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:14.132348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:14.134966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:14.135322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:14.135667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] PATH_INFO: `/v3/users/abe58e23a61e4e049782593e95002d48` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:14.137527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] RBAC: Authorizing `identity:get_user(user_id=abe58e23a61e4e049782593e95002d48)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:14.139515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:14.141867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 161/314] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:13 2022] GET /identity/v3/users/abe58e23a61e4e049782593e95002d48 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:14.147779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:14.166682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c78b-79e0-4b36-8582-a18ee038aff9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:14.203242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:14.203904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:14.204190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:14.204472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] PATH_INFO: `/v3/system/users/2d0eab97bba841b79b2526391559a157/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:14.205963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d0eab97bba841b79b2526391559a157, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:14.207520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:14.219559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 154/315] 10.222.0.81 () {58 vars in 1334 bytes} [Wed Aug 24 03:17:14 2022] PUT /identity/v3/system/users/2d0eab97bba841b79b2526391559a157/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:14.291689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1f31b-a29b-42f5-b07d-78a2e5f7bc5c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:14.333900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:14.334513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:14.334785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:14.335049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:14.340137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:14.412945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:14.415033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:14.525493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 162/316] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:17:14.531148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:14.629828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:14.638705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62029a-178c-43ca-88a8-636d6b5f6a06 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:14.684672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58682f44-87b1-401c-88c5-3ad594d61824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:14.685714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58682f44-87b1-401c-88c5-3ad594d61824 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:14.686189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58682f44-87b1-401c-88c5-3ad594d61824 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:14.686657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58682f44-87b1-401c-88c5-3ad594d61824 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:14.687999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58682f44-87b1-401c-88c5-3ad594d61824 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:14.689724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58682f44-87b1-401c-88c5-3ad594d61824 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:14.691974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 155/317] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:14 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:14.698385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:14.736631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:14.806711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb25164-aedc-47ee-adaa-a6b7f7a14556 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:14.841852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:14.842460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:14.842729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:14.842993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:14.843723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] RBAC: Adding query filter params (role.id=ed708940f2d64bfab650ce5deae860cd, scope.project.id=0d1859c09d264db597aab109f2d67e77, user.id=abe58e23a61e4e049782593e95002d48) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:14.844023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:14.845551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd5d11-881d-433d-a3fb-45133b5757ff None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:14.856579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 163/318] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:14 2022] GET /identity/v3/role_assignments?user.id=abe58e23a61e4e049782593e95002d48&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=ed708940f2d64bfab650ce5deae860cd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:15.321286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e942a8-63dc-4178-8536-92f9a1577ebf None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:15.321626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e942a8-63dc-4178-8536-92f9a1577ebf None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:15.321813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e942a8-63dc-4178-8536-92f9a1577ebf None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:15.323273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 156/319] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:15.328321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:15.328756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:15.328952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:15.348401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:15.349642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:15.418497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 164/320] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:15.684979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e942a8-63dc-4178-8536-92f9a1577ebf None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:15.790484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e942a8-63dc-4178-8536-92f9a1577ebf None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:15.823901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2b69b-c8e5-471d-b40c-38bec9472475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:15.824521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2b69b-c8e5-471d-b40c-38bec9472475 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:15.824810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2b69b-c8e5-471d-b40c-38bec9472475 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:15.825080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2b69b-c8e5-471d-b40c-38bec9472475 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:15.826155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2b69b-c8e5-471d-b40c-38bec9472475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:15.827619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2b69b-c8e5-471d-b40c-38bec9472475 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:15.829398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 157/321] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:15.835829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:15.867956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:15.960997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ca08e-12d8-4e32-b428-ba5c4c2001ce None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:15.997209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6091cc-f3f2-4b70-b8a7-7d48656be9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:15.997819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6091cc-f3f2-4b70-b8a7-7d48656be9fb None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:15.998096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6091cc-f3f2-4b70-b8a7-7d48656be9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:15.998368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6091cc-f3f2-4b70-b8a7-7d48656be9fb None admin] PATH_INFO: `/v3/groups` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:15.998999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6091cc-f3f2-4b70-b8a7-7d48656be9fb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:16.000447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6091cc-f3f2-4b70-b8a7-7d48656be9fb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:16.004287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab8270-f648-4b83-9fd4-be071c462dff None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.004591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab8270-f648-4b83-9fd4-be071c462dff None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.004885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab8270-f648-4b83-9fd4-be071c462dff None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.007007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 158/322] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:16.012330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00beb-4f57-4956-ba3a-efa71901c368 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.012619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00beb-4f57-4956-ba3a-efa71901c368 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.012927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e00beb-4f57-4956-ba3a-efa71901c368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.020977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 165/323] 10.222.0.81 () {60 vars in 1134 bytes} [Wed Aug 24 03:17:15 2022] POST /identity/v3/groups => generated 226 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:17:16.035736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e7e00beb-4f57-4956-ba3a-efa71901c368 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:16.037045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e7e00beb-4f57-4956-ba3a-efa71901c368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:16.065006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:16.116844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 159/324] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:16.193036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3d423-5d09-4182-952d-64532d4f07d9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.193991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3d423-5d09-4182-952d-64532d4f07d9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.194276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3d423-5d09-4182-952d-64532d4f07d9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.195807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 166/325] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:16.205705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.206022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.206261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.226871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:16.227992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:16.302144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 160/326] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:16.399236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3d423-5d09-4182-952d-64532d4f07d9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:16.501354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:16.514022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3d423-5d09-4182-952d-64532d4f07d9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:16.550087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:16.550765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.551047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.551319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.563358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:16.568494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:16.569549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:16.570325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 167/327] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:16.576201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:16.592301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:16.691196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:16.705645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0465db-2f58-4d8d-9848-600585c53cd3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:16.727066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594fa39-51eb-4c08-8319-8ed45b2c608c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:16.755902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:16.756739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.757180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.757548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.758318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:16.758714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:16.760190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:16.769372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:16.770015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.770553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.771360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.773490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 161/328] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:16.779849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:16.783606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:16.789985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:16.790868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:16.791634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:16.792229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 168/329] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:16.797710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:16.813840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:16.915427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2bc9e-fa6f-43df-9013-f4ca38ed6dd6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:16.927054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece7bab-5651-4a25-b22f-eb058e4d04b3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:16.953611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:16.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.954585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.954855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.955617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:16.955914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:16.957428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:16.967462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 169/330] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:16.972848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:16.973311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:16.974256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:16.974610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:16.974881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] PATH_INFO: `/v3/users/babfe08cd0944213b282bba8cf3ca3fc` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:16.977264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] RBAC: Authorizing `identity:get_user(user_id=babfe08cd0944213b282bba8cf3ca3fc)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:16.978892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:16.980891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 162/331] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:16 2022] GET /identity/v3/users/babfe08cd0944213b282bba8cf3ca3fc => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:16.986686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:17.009520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:17.072235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:17.082332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af943-54ef-45a5-9711-7e9b397986d0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:17.099600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ada083-9a00-489f-a4ff-e4565e170fbd None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:17.121861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:17.123257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:17.123257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:17.123257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] PATH_INFO: `/v3/users/2d0eab97bba841b79b2526391559a157` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:17.124240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] RBAC: Authorizing `identity:get_user(user_id=2d0eab97bba841b79b2526391559a157)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:17.125883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:17.127848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 170/332] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:16 2022] GET /identity/v3/users/2d0eab97bba841b79b2526391559a157 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:17.133721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:17.141641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b64e8-94cc-495a-965d-19c378068f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:17.142573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b64e8-94cc-495a-965d-19c378068f77 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:17.143862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b64e8-94cc-495a-965d-19c378068f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:17.144359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b64e8-94cc-495a-965d-19c378068f77 None admin] PATH_INFO: `/v3/system/users/babfe08cd0944213b282bba8cf3ca3fc/roles/ed708940f2d64bfab650ce5deae860cd` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:17.146199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b64e8-94cc-495a-965d-19c378068f77 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=babfe08cd0944213b282bba8cf3ca3fc, role_id=ed708940f2d64bfab650ce5deae860cd)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:17.147767 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b64e8-94cc-495a-965d-19c378068f77 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:17.165042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 163/333] 10.222.0.81 () {58 vars in 1334 bytes} [Wed Aug 24 03:17:16 2022] PUT /identity/v3/system/users/babfe08cd0944213b282bba8cf3ca3fc/roles/ed708940f2d64bfab650ce5deae860cd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:17:17.276288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1f35a-37b2-4a64-949d-2fde4027b932 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:17.320362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:17.321009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:17.321285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:17.321553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:17.322330 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] RBAC: Adding query filter params (role.id=fed56dd461254bc682cf1955076ea721, user.id=2d0eab97bba841b79b2526391559a157, scope.system=all) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:17.322857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:17.324376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd5931-be33-4ddd-9384-52945898981f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:17.331302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:17.339780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 171/334] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Aug 24 03:17:17 2022] GET /identity/v3/role_assignments?user.id=2d0eab97bba841b79b2526391559a157&scope.system=all&role.id=fed56dd461254bc682cf1955076ea721 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:18.316816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355d73-0658-4352-a62a-0fb0a9a5aa26 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:18.317801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355d73-0658-4352-a62a-0fb0a9a5aa26 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:18.318303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355d73-0658-4352-a62a-0fb0a9a5aa26 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:18.320123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 164/335] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:18.326207 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd497f-86ce-42b0-84f5-8328578bf34c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:18.327054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd497f-86ce-42b0-84f5-8328578bf34c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:18.327527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd497f-86ce-42b0-84f5-8328578bf34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:18.353662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d4bd497f-86ce-42b0-84f5-8328578bf34c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:18.355210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d4bd497f-86ce-42b0-84f5-8328578bf34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:18.442789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 172/336] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:18.702264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bf66b-4b18-494b-9ac1-67efbbcea7f2 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:18.702644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bf66b-4b18-494b-9ac1-67efbbcea7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:18.702819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bf66b-4b18-494b-9ac1-67efbbcea7f2 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:18.704198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 165/337] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:18.710095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:18.710383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:18.710654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:18.730636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:18.731833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:18.806237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bf66b-4b18-494b-9ac1-67efbbcea7f2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:18.825042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 173/338] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:18.939994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:18.961169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bf66b-4b18-494b-9ac1-67efbbcea7f2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:19.001589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:19.002218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:19.002502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:19.002773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:19.021802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:19.026853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:19.027742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:19.028523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 166/339] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:19.036151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:19.118677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:19.168348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36b4e-9322-41cd-a731-29b3b3f66b69 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:19.209886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:19.210493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:19.210764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:19.211014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:19.211753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:19.212000 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:19.213483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:19.223865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 174/340] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:19.234098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:19.380241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:19.403135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e63c5d-42e0-4ef3-9fca-68b518776c2f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:19.455551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:19.456273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:19.456551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:19.456851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:19.477288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:19.534761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44c9b4-d988-4c6c-ae82-6bf6fcbc78b3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:19.557223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:19.589782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:19.591241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:19.593429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:19.594058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:19.594329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:19.594594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:19.615892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:19.624138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:19.625113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:19.625886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:19.626508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 175/341] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:19.635213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:19.795356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 167/342] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:19.808412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:19.812595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88f4a4-54e7-482f-b42a-f59ff940146f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:19.871124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:19.872281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:19.872797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:19.873271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:19.874143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:19.874674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:19.876264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:19.895668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:19.896573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 176/343] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:19.902508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:19.993826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:19.995348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0b615-a2e6-43f7-bbff-c47c2c1a6c0c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:20.069618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:20.070143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.070426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.070701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.072006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:20.073489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:20.075427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 168/344] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:19 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:20.080791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:20.094312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc447c4-c931-433f-9680-f39c7e7be694 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:20.168740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf27db9-7c25-4abb-9087-d0af10768023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:20.169415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf27db9-7c25-4abb-9087-d0af10768023 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.169691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf27db9-7c25-4abb-9087-d0af10768023 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.169959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf27db9-7c25-4abb-9087-d0af10768023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.250925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88219b00-227a-47cc-8619-eef52a691657 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:20.266203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:20.305535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf27db9-7c25-4abb-9087-d0af10768023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:20.307001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf27db9-7c25-4abb-9087-d0af10768023 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:20.320097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ddce2-7a30-4482-9c5f-e38067b2fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:20.320658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ddce2-7a30-4482-9c5f-e38067b2fc31 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.320948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ddce2-7a30-4482-9c5f-e38067b2fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.321217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ddce2-7a30-4482-9c5f-e38067b2fc31 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.330563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ddce2-7a30-4482-9c5f-e38067b2fc31 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:20.332028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ddce2-7a30-4482-9c5f-e38067b2fc31 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:20.335729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 169/345] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:20 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:20.341074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b9e12-a91f-4f36-9f3f-88d7a2233579 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.341377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b9e12-a91f-4f36-9f3f-88d7a2233579 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.341651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b9e12-a91f-4f36-9f3f-88d7a2233579 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.343023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 170/346] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:20.344087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ade817-f408-41e8-8f0e-b325eb11ae7b None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.344365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ade817-f408-41e8-8f0e-b325eb11ae7b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.344694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ade817-f408-41e8-8f0e-b325eb11ae7b None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.346095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 171/347] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:20.351637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.351917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.352188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.371688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:20.386832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:20.388524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:20.452866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 177/348] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:17:20.452866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.452866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.452866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.471382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:20.472646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:20.484282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 172/349] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:20.486958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:20.592982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 178/350] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:20.593747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:20.673198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:20.699216 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8f51c-be6b-42eb-b03c-93339449fd58 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:20.704247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:20.726925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfc75e-6c1b-4dd8-9694-670cc99ef953 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:20.771406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:20.772304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.772749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.773203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.774872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=1b7cc325ca0144afbe3094beaaf32144) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:20.777236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:20.778853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d047c-4137-4c70-b68f-ab7fafcbae20 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:20.798245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 173/351] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:20 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=fed56dd461254bc682cf1955076ea721 => generated 260 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:20.805626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:20.806577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.807025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.807503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.809439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:20.811112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:20.812975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 179/352] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:20 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:20.815068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc973-50b9-45aa-bdf5-1e0722ca3397 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.815068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc973-50b9-45aa-bdf5-1e0722ca3397 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.815068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc973-50b9-45aa-bdf5-1e0722ca3397 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.816380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 174/353] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:20.822255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:20.827205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:20.827490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:20.827771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:20.856999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:20.858289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:20.951234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 175/354] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:20.952193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.074701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7039e-db19-4296-955d-e7119dc160af None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.115932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:21.125275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.127623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.128598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.129620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.129941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a60e0-dae3-4d03-87f9-cdf07efb3fe1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.132883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:21.134730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:21.137117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 180/355] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:20 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:21.138102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.156716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:21.213414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.214413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.214925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.215494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.234161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:21.242828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:21.243799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:21.244680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:21.248854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 176/356] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:21.249944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.317826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b40bb-ff2f-4407-8c10-6ffd3cac2dcc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.370914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c7284-c6d2-46d7-ae19-0fff22db6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.371475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c7284-c6d2-46d7-ae19-0fff22db6207 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.371750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c7284-c6d2-46d7-ae19-0fff22db6207 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.372014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c7284-c6d2-46d7-ae19-0fff22db6207 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.386816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394b6d-e853-415d-8f9d-01cbf4483eb7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.391071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c7284-c6d2-46d7-ae19-0fff22db6207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e88c7284-c6d2-46d7-ae19-0fff22db6207 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:21.396305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:21.397325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:17:21.398168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:21.398815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 181/357] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:21.402087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d4baf-71c4-40fd-8594-5b5ddc3b345f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.402087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d4baf-71c4-40fd-8594-5b5ddc3b345f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.402087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d4baf-71c4-40fd-8594-5b5ddc3b345f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.402087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 182/358] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:21.402087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d4baf-71c4-40fd-8594-5b5ddc3b345f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.428190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.428812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.429091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.429355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.430081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=1b7cc325ca0144afbe3094beaaf32144) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:21.430382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:21.433798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:21.441455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 177/359] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:21.443565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.481143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:21.506009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:21.561218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d4baf-71c4-40fd-8594-5b5ddc3b345f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.588469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1097f-2611-4227-83ba-a8e47df0f985 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.617144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.618420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.618827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.619218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.620012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:21.620430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:21.622064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:21.636251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 183/360] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:21.638260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.656153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec8ec7-d962-4923-beb2-4c6978f40a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.657115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec8ec7-d962-4923-beb2-4c6978f40a5f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.658021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec8ec7-d962-4923-beb2-4c6978f40a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.658923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec8ec7-d962-4923-beb2-4c6978f40a5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.674034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec8ec7-d962-4923-beb2-4c6978f40a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-3bec8ec7-d962-4923-beb2-4c6978f40a5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:21.681934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:21.682972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:21.683950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:21.686652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 178/361] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:21.687948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.688319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.688954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.711905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:21.713650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:21.778771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5c5d3-11be-444a-b0e5-bed3dad34126 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:21.800700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 179/362] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:21.801640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:21.819203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:21.828625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:21.840413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:21.841055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:21.841332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:21.841604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:21.843357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:21.843661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:21.845549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:21.873175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 184/363] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:21.874001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.052787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97acb2-aaa9-4824-8318-5a716728f87a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.067646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ebdd-e6b7-4ac2-ad46-747c48a7db53 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.099253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.102155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.102963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.103253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.103930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:22.104235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:22.106111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:22.117389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 185/364] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:22.121519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.169665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.170213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.170491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.170762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.205526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:22.253868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333cb24-43b0-4aea-89f3-a7f7fed7fb10 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.258250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:22.259724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:22.274503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:22.292719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.293376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.293649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.293916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] PATH_INFO: `/v3/users/babfe08cd0944213b282bba8cf3ca3fc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.295051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] RBAC: Authorizing `identity:get_user(user_id=babfe08cd0944213b282bba8cf3ca3fc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:22.296483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:22.306352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 186/365] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/users/babfe08cd0944213b282bba8cf3ca3fc => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:22.306352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.415096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 180/366] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:22.416032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.436022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ceefc-5b2a-4d67-933f-a1503fe36904 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.462339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:22.475628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.476244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.476524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.476810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.539617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efcb4-34e5-4977-a238-dd15afe01f01 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.558248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:22.559131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:22.561818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:22.587908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.587908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.587908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.587908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.605101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:22.609961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:22.610892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:22.611739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 181/367] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:22.612445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.702203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 187/368] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:22.704932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.732616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:22.752021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49707596-294e-4dba-bca8-150797bc0a20 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.796623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.797313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.797593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.797868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.798604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] RBAC: Adding query filter params (role.id=ed708940f2d64bfab650ce5deae860cd, user.id=babfe08cd0944213b282bba8cf3ca3fc, scope.system=all) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:22.798911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:22.800373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:22.823514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 182/369] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/role_assignments?user.id=babfe08cd0944213b282bba8cf3ca3fc&scope.system=all&role.id=ed708940f2d64bfab650ce5deae860cd => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:22.824558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.839172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fc0ee-05ee-4efb-9773-05c54ea8ab1e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:22.884020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:22.885012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:22.885430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:22.885910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:22.887287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:22.888918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:22.891461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 188/370] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:22.892493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:22.914144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.011157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.023534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f445631-2222-4c32-bf12-f2bdceff03ef None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.072193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.073197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.074180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.074614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.075929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.077558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.084526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 189/371] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:23.085626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.095669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea87d3-aa34-44bf-8374-4cfbc58094cc None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.137016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.137637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.137917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.138188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.138866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:23.139170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.140676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.150626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 183/372] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:23.151513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.202351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a2486-3b37-495e-abcd-1842d6bf5cac None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.244371 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.245042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.245318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.245590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.246736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.248638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.251051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 190/373] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:23.251760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.272233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.295353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b13e40-5641-4c30-b444-b534bba59654 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.300059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.351801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911e435-8983-4815-9941-a6d97e4ea199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.352433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911e435-8983-4815-9941-a6d97e4ea199 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.352714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911e435-8983-4815-9941-a6d97e4ea199 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.353054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911e435-8983-4815-9941-a6d97e4ea199 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.355188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911e435-8983-4815-9941-a6d97e4ea199 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.356650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911e435-8983-4815-9941-a6d97e4ea199 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.358435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 184/374] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:23.363301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6cb38-4ab0-4305-a06a-578388c42ca4 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.363301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6cb38-4ab0-4305-a06a-578388c42ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.363301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6cb38-4ab0-4305-a06a-578388c42ca4 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.363301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 185/375] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:23.363301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6cb38-4ab0-4305-a06a-578388c42ca4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.407466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8774c-dd5f-4207-8e4c-d6dcee9e5d99 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.455526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.455745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.455891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.456737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.491899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6cb38-4ab0-4305-a06a-578388c42ca4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.531263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.532193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.532492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.533729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.534509 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=0d1859c09d264db597aab109f2d67e77) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:23.534821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.536259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb6b10-cd39-44a3-8bac-7dc3316f2a54 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.543981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.544605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 186/376] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=fed56dd461254bc682cf1955076ea721 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:23.545508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.545816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.546163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.547010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.572017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:23.573667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:23.594372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.622894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.665689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 187/377] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:23.666974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.692305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 191/378] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:23.698123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.832755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a9562-b2c0-47a3-a1c0-f43c6bc13264 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.854485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2d4e-4063-4eca-9553-44f84b41e0b6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:23.877955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.879454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.879454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.879639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.880325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=0d1859c09d264db597aab109f2d67e77) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:23.880632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.882478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c9660-3196-425f-9582-658b281a01d4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.886284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:23.896793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 188/379] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:23.896793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc74091-701f-464e-9312-3d0b31b8e047 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.896793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc74091-701f-464e-9312-3d0b31b8e047 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.896793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc74091-701f-464e-9312-3d0b31b8e047 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.897188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 189/380] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:23.903229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.903519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.903774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.911392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:23.912454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:23.912960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:23.913424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] PATH_INFO: `/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:23.920815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=35c28eab9a294b9ba17b7f5b9c24d868)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:23.923165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:23.925615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 192/381] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868 => generated 344 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:23.930221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:23.931166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:23.931452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:23.949644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:24.040969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 190/382] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:24.042833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.093184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12bdd7-d720-454d-a181-88fa38c57aa6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.149402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.150408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.150920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.151441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] PATH_INFO: `/v3/groups/f8d794d38a9b47d8a91f3a37ccda8548` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.159619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] RBAC: Authorizing `identity:get_group(group_id=f8d794d38a9b47d8a91f3a37ccda8548)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:24.162295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:24.164626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 193/383] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:23 2022] GET /identity/v3/groups/f8d794d38a9b47d8a91f3a37ccda8548 => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:24.182386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f465bc9-bd27-4e43-a401-f300c11b036d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.188017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.239920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.240556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.240851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.241157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.257726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:24.262512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:24.263464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:24.264240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 191/384] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:24.267506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:24.278058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.292183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:24.321360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9666bc-5f6b-4486-a36b-6619ee7c1567 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.369345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.370053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.370333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.370601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.371335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] RBAC: Adding query filter params (group.id=f8d794d38a9b47d8a91f3a37ccda8548, role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=35c28eab9a294b9ba17b7f5b9c24d868) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:24.371632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:24.373201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:24.383868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 194/385] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/role_assignments?group.id=f8d794d38a9b47d8a91f3a37ccda8548&scope.project.id=35c28eab9a294b9ba17b7f5b9c24d868&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:24.408842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.423817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fbba3-c6e9-432a-9b11-94362e770092 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.470074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.470704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.470982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.471257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.471944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] RBAC: Adding query filter params (name=member) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:24.472248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:24.473696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:24.490815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 192/386] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:24.498024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.629430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:24.638116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c040-0193-4ff7-810c-efd67f928a3a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.653075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6266de-8de9-4cff-a019-9215f06be886 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.690928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.691561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.691841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.692116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.693301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:24.694756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:24.696704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 193/387] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:24.698982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:24.704550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.744367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.745024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.745298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.745564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.781844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:24.787081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:24.787962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:24.788815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:24.797460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 195/388] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:24.798398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.818772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a02dad-c20b-414b-ba81-eebee8c5b42c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:24.856422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:24.857344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:24.857925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:24.859035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:24.860546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:24.862244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:24.864612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 194/389] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:24.871958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:24.884082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:24.990728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042609eb-e1a8-47aa-abdf-d1c41dd395ec None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:25.028053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:25.028716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:25.029023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:25.029302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144/groups/e49e2afcb344451bb1ce073aee63a862/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:25.032229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b7cc325ca0144afbe3094beaaf32144, group_id=e49e2afcb344451bb1ce073aee63a862, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:25.033732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:25.050578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 195/390] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Aug 24 03:17:24 2022] PUT /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144/groups/e49e2afcb344451bb1ce073aee63a862/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:17:25.088034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c6fff-91b6-4ea1-bfec-ae44abcea699 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:25.143987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:25.144521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:25.144819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:25.145088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:25.145758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:25.146102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:25.147572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:25.158393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 196/391] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:25.163755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:25.264867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:25.305923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b05f7-49ca-4c09-b307-8f11cb1d74c6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:25.348304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:25.348980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:25.349254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:25.349522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:25.350650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:25.352088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:25.353982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 196/392] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:25 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:25.360393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:25.419791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:25.487192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c8fb6-18a7-49da-bd96-e887aaed6738 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:25.528730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce543cf-25b1-40b2-a4ec-afbd7804876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:25.529382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce543cf-25b1-40b2-a4ec-afbd7804876c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:25.529674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce543cf-25b1-40b2-a4ec-afbd7804876c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:25.529946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce543cf-25b1-40b2-a4ec-afbd7804876c None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:25.531101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce543cf-25b1-40b2-a4ec-afbd7804876c None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:25.532539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce543cf-25b1-40b2-a4ec-afbd7804876c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:25.534176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 197/393] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:25 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:25.539809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:25.665489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb08c8e-8d9f-4e76-a16e-993101708573 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:25.707883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8de297-ce27-4449-aebe-12171a744776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:25.708469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de297-ce27-4449-aebe-12171a744776 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:25.708741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de297-ce27-4449-aebe-12171a744776 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:25.709036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de297-ce27-4449-aebe-12171a744776 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144/groups/e49e2afcb344451bb1ce073aee63a862/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:25.711180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de297-ce27-4449-aebe-12171a744776 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b7cc325ca0144afbe3094beaaf32144, group_id=e49e2afcb344451bb1ce073aee63a862, role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:25.712629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de297-ce27-4449-aebe-12171a744776 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:25.714988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:25.733481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 197/394] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Aug 24 03:17:25 2022] PUT /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144/groups/e49e2afcb344451bb1ce073aee63a862/roles/7eef59c95b8945768aab3374bc2721f1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:26.270221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98cba-70d6-4d5c-9d8e-13a02a09a79d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:26.270755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98cba-70d6-4d5c-9d8e-13a02a09a79d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:26.270755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98cba-70d6-4d5c-9d8e-13a02a09a79d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:26.272375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 198/395] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:26.279438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968718ad-9637-4934-8efc-2951ff88299e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:26.279741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968718ad-9637-4934-8efc-2951ff88299e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:26.280021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968718ad-9637-4934-8efc-2951ff88299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:26.313866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-968718ad-9637-4934-8efc-2951ff88299e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:26.315094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-968718ad-9637-4934-8efc-2951ff88299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:26.412116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 198/396] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:26.600885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28997312-7e14-4e1b-9a16-67d31ce56c28 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:26.600885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28997312-7e14-4e1b-9a16-67d31ce56c28 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:26.600885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28997312-7e14-4e1b-9a16-67d31ce56c28 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:26.602465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 199/397] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:26.614504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb051bd-362b-4913-a7e8-55cd3f4c5ea0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:26.614792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb051bd-362b-4913-a7e8-55cd3f4c5ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:26.615093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb051bd-362b-4913-a7e8-55cd3f4c5ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:26.639397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-4bb051bd-362b-4913-a7e8-55cd3f4c5ea0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:26.640651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4bb051bd-362b-4913-a7e8-55cd3f4c5ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:26.737617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 199/398] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:26.994039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28997312-7e14-4e1b-9a16-67d31ce56c28 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.086586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c2830-bb1f-41ca-8806-d05657903585 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.086923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c2830-bb1f-41ca-8806-d05657903585 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.087172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c2830-bb1f-41ca-8806-d05657903585 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.088807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 200/399] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:27.094890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.095178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.095463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.116997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28997312-7e14-4e1b-9a16-67d31ce56c28 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.130647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:27.136034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:27.137616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:27.152577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.153226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.154249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.154527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.165078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:27.169688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:27.170558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:27.171334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 200/400] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:27.171975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.205404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:27.229037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 201/401] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 03:17:27.230616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.292984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd650-f684-493d-bab2-eff8068cf041 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.347142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.347765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.348037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.348303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.357476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd07be-2273-4cb2-aec8-d0b8b7d62b06 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.362202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:27.366976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:27.367844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:27.368656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 201/402] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:27.374442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.413163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:27.419638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.422403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.422403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.422403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.425436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] RBAC: Adding query filter params (name=member) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:27.425746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:27.427138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:27.449050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 202/403] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:27.455284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.523774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:27.566896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc8003-71e3-4372-afb4-01ff46cb192e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.588483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04499d0-6225-4609-8a11-ba14e5e07a63 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.633275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.633901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.634173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.634443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.638663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:27.641288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:27.643403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 203/404] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:27.648908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.649920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.650408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.650884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.650992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.651871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:27.652363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:27.654985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:27.667192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 202/405] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:27.668303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.767205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e607fa9-b1b8-4b1a-a288-8853c06c23d9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.781427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:27.804657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:27.804892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.805771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.805771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.806038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.807366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:27.809193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:27.815867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 204/406] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:27.815867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.821456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ff5f2-8999-426d-9ffc-6a44ded00c2f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.872075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.873403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.873882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.874519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.892796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:27.898128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:27.898993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:27.899782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 203/407] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:27.900472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:27.931367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1ff0e-62a7-4514-bd77-9a30d207fdfa None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:27.968711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fa98a-5f77-4f4d-806d-7f95d0b98ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:27.969381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fa98a-5f77-4f4d-806d-7f95d0b98ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.969656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fa98a-5f77-4f4d-806d-7f95d0b98ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.969928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fa98a-5f77-4f4d-806d-7f95d0b98ea8 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.971040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fa98a-5f77-4f4d-806d-7f95d0b98ea8 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:27.972499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fa98a-5f77-4f4d-806d-7f95d0b98ea8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:27.978615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 205/408] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:27.978615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557c67-f2b3-41ac-ab7b-62de4c57eb5e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:27.978615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557c67-f2b3-41ac-ab7b-62de4c57eb5e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:27.978615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557c67-f2b3-41ac-ab7b-62de4c57eb5e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:27.981679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 206/409] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:27.982602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2557c67-f2b3-41ac-ab7b-62de4c57eb5e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.018294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2c46-0de2-4f9b-8e97-51833c1a7fde None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.039021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:28.057728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.058412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.058694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.058995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77/groups/e49e2afcb344451bb1ce073aee63a862/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.061407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] RBAC: Authorizing `identity:create_grant(project_id=0d1859c09d264db597aab109f2d67e77, group_id=e49e2afcb344451bb1ce073aee63a862, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.062952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:28.088791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 204/410] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Aug 24 03:17:27 2022] PUT /identity/v3/projects/0d1859c09d264db597aab109f2d67e77/groups/e49e2afcb344451bb1ce073aee63a862/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:28.088791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.108339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2557c67-f2b3-41ac-ab7b-62de4c57eb5e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.108878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:28.162278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.163166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.163951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.164238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.165246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:28.165548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.167771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f94dd-09ce-401c-bc32-cdd0f33f7803 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:28.182552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 207/411] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:28.183939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813827-e51b-490e-8f31-64882207291e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.184865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813827-e51b-490e-8f31-64882207291e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.185140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813827-e51b-490e-8f31-64882207291e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.207868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a1813827-e51b-490e-8f31-64882207291e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:28.209182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a1813827-e51b-490e-8f31-64882207291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:28.223515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df4cb-a012-44e9-97f6-71f611604d5f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.268198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.269429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.269660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.270270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.271653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.273527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:28.280848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 205/412] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:28 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:28.280848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.289679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 208/413] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:28.292870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dd93d-033f-4ed0-ae49-ba6b2d881c62 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.292870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dd93d-033f-4ed0-ae49-ba6b2d881c62 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.292870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dd93d-033f-4ed0-ae49-ba6b2d881c62 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.292870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 209/414] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:28.293813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dd93d-033f-4ed0-ae49-ba6b2d881c62 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.346961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:28.391424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:28.413668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dd93d-033f-4ed0-ae49-ba6b2d881c62 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.417264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84341cf2-9444-4dab-bcc8-18c9b776e4b8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.452408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3fdce-3c28-4224-96b2-239e3785af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.453526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3fdce-3c28-4224-96b2-239e3785af17 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.454070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3fdce-3c28-4224-96b2-239e3785af17 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.454569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3fdce-3c28-4224-96b2-239e3785af17 None admin] PATH_INFO: `/v3/groups/f8d794d38a9b47d8a91f3a37ccda8548` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.456101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3fdce-3c28-4224-96b2-239e3785af17 None admin] RBAC: Authorizing `identity:get_group(group_id=f8d794d38a9b47d8a91f3a37ccda8548)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.456255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.457070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.457414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.457755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77/groups/e49e2afcb344451bb1ce073aee63a862/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.458588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3fdce-3c28-4224-96b2-239e3785af17 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:28.460055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] RBAC: Authorizing `identity:create_grant(project_id=0d1859c09d264db597aab109f2d67e77, group_id=e49e2afcb344451bb1ce073aee63a862, role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.461042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 206/415] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:28 2022] GET /identity/v3/groups/f8d794d38a9b47d8a91f3a37ccda8548 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:28.462425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.462867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.463295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.467040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:28.484351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:28.485897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:28.492983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 210/416] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Aug 24 03:17:28 2022] PUT /identity/v3/projects/0d1859c09d264db597aab109f2d67e77/groups/e49e2afcb344451bb1ce073aee63a862/roles/7eef59c95b8945768aab3374bc2721f1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:28.494699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.585404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 207/417] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:28.595463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.712597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842a16a-9dd4-48dc-af39-c901c36fef6a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.714477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:28.726960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:28.754082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb472a-0f63-40cd-b9af-4a9d96e2a303 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.760791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.761458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.761771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.762566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] PATH_INFO: `/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.763768 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] RBAC: Authorizing `identity:get_project(project_id=35c28eab9a294b9ba17b7f5b9c24d868)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.765702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:28.768334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 211/418] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:28 2022] GET /identity/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868 => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:28.774076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.792110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.792736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.793550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.793820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.804833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:28.811836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:28.812734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:28.813595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 208/419] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:28.819190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:28.924970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ed67-edb5-431f-aa59-d7e24126db7f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.969429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563464-a398-4800-829a-742359bee83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:28.970289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563464-a398-4800-829a-742359bee83e None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:28.970560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8872b9-a11d-44d9-bef5-63b5cdb49e46 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:28.970753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563464-a398-4800-829a-742359bee83e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:28.971128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563464-a398-4800-829a-742359bee83e None admin] PATH_INFO: `/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868/groups/f8d794d38a9b47d8a91f3a37ccda8548/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:28.974315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563464-a398-4800-829a-742359bee83e None admin] RBAC: Authorizing `identity:create_grant(project_id=35c28eab9a294b9ba17b7f5b9c24d868, group_id=f8d794d38a9b47d8a91f3a37ccda8548, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:28.975872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563464-a398-4800-829a-742359bee83e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:29.000820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 212/420] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Aug 24 03:17:28 2022] PUT /identity/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868/groups/f8d794d38a9b47d8a91f3a37ccda8548/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:29.002061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563464-a398-4800-829a-742359bee83e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:29.028071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.028810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.029087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.029410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.030088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:29.031424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:29.035440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:29.053786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 209/421] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:29.060059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:29.069993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:29.079420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:29.174516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563464-a398-4800-829a-742359bee83e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:29.219622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.220287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.220925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.221207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.236971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:29.243429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:29.245474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:29.247166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 213/422] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:29.254438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:29.286532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff935ae-446b-4f31-9cf1-5b3fd685f905 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:29.335848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.337972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.338252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.338518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.387063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:29.393668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d8dfc-c08e-4b19-8e0f-99ac3c476d44 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:29.448392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.449476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.449744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.450008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.450672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:29.450966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:29.452342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:29.467020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 214/423] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:29.473200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:29.494482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:29.496034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:29.498236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:29.648423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 210/424] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:29.649737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:29.712453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92865c5a-670c-4079-a288-a4ce45060516 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:29.760593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9b916-797c-40b7-b845-d87b53f8f21d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:29.777840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:29.801313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.801962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.802231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.802497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.803847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:29.805796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:29.812965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 211/425] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:29.815145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:29.818382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.819172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.819452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.819767 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.883097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:29.942323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6138483-6b42-4d19-8cef-e658df67ecd4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:29.979946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:29.980540 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:29.980861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:29.981483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:29.982624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:29.984134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:29.991160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 212/426] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:29.992359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:30.037697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:30.038422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:30.039893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:30.104688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8efa62-4e6c-4aa5-8674-a8aa67e68e97 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:30.161958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:30.162593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.162878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.163156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.163922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=1b7cc325ca0144afbe3094beaaf32144) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:30.164229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:30.165770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12a00b-f088-4206-8fe4-9a88e66ebc30 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:30.181416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 213/427] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=fed56dd461254bc682cf1955076ea721 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:30.275734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e669d-048f-4478-94a7-658db9d4c31c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.275919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e669d-048f-4478-94a7-658db9d4c31c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.276206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e669d-048f-4478-94a7-658db9d4c31c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.278712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 214/428] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:30.284286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.284576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.284882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.287354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 215/429] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 815 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:17:30.294382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:30.311050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:30.312631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:30.390128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:30.426673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 215/430] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:30.453431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273de93-508a-463e-8c5a-65d83802ab0f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:30.494578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:30.495802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.496125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.496591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] PATH_INFO: `/v3/projects/1b7cc325ca0144afbe3094beaaf32144` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.497906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] RBAC: Authorizing `identity:get_project(project_id=1b7cc325ca0144afbe3094beaaf32144)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:30.499360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:30.501494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 216/431] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:30 2022] GET /identity/v3/projects/1b7cc325ca0144afbe3094beaaf32144 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:30.506620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:30.549379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:30.634699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d7b76-5ebb-4ef5-ae56-d85f615b5e9c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:30.676088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9d3ae-8fda-4fa8-981c-f5c44c77a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:30.676719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9d3ae-8fda-4fa8-981c-f5c44c77a22a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.677025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9d3ae-8fda-4fa8-981c-f5c44c77a22a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.677301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9d3ae-8fda-4fa8-981c-f5c44c77a22a None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.678412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9d3ae-8fda-4fa8-981c-f5c44c77a22a None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:30.679871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9d3ae-8fda-4fa8-981c-f5c44c77a22a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:30.681760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 216/432] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:30 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:30.687701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:30.773642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc0f63-18c5-4a5c-8702-b3aa90b14637 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.774261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc0f63-18c5-4a5c-8702-b3aa90b14637 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.774742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc0f63-18c5-4a5c-8702-b3aa90b14637 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.777199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 217/433] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:30.782608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.782883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.783154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.806277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:30.807403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:30.827355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273631a-e1c7-4c2c-a1f6-73864f077c4f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:30.851635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:30.877540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:30.878142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:30.878414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:30.878681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:30.879410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=1b7cc325ca0144afbe3094beaaf32144) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:30.879710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:30.882466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:30.891912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 217/434] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:30 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=1b7cc325ca0144afbe3094beaaf32144&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:30.891912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:30.914591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 218/435] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:31.009365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c13eef-dcc7-4be5-84db-5b6cb937829a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.049741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.050397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.050672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.050944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.063652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:31.069036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:31.069970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:31.070745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 218/436] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:31.076950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.116632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.201173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b106d9-f4e8-48d8-8633-2b67ad9c2137 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.213256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.241100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.241735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.241969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.242236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.242898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:31.243193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:31.244562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:31.255174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 219/437] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:31.259921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.282591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.352302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5bfb0-6d22-498a-b34f-763902fe591d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.383805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.393826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481c72d-afe2-4569-99b8-50caf644ea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.394544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481c72d-afe2-4569-99b8-50caf644ea34 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.394820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481c72d-afe2-4569-99b8-50caf644ea34 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.395092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481c72d-afe2-4569-99b8-50caf644ea34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.395602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733f9aa-3d75-4769-965d-0bdc96efa559 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.406434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481c72d-afe2-4569-99b8-50caf644ea34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-3481c72d-afe2-4569-99b8-50caf644ea34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:31.411693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:31.412618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:31.413428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 219/438] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:31.414129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b003273-eab8-4c9d-99c1-f58b0b517eb0 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.414409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b003273-eab8-4c9d-99c1-f58b0b517eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.414687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b003273-eab8-4c9d-99c1-f58b0b517eb0 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.418981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 220/439] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:31.421253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b003273-eab8-4c9d-99c1-f58b0b517eb0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.437551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.438162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.438437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.438713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.513318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:31.514785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:31.529543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b003273-eab8-4c9d-99c1-f58b0b517eb0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.532117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.571359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.571966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.572241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.572509 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.573424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:31.573718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:31.575092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca013-1407-4753-ab19-f76f2cc2a3c0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:31.583421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 221/440] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:31.584592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.584909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.585185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.606083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:31.607769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:31.624216 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.633774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 220/441] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:31.634570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.682632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 222/442] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:31.683832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.765622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47782459-c30c-4b2e-8042-2a147a23ea99 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.808730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.817830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65710b32-c52f-4b35-a837-9307d62ac8ca None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:31.819316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.819929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.820367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.820645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.860842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:31.861512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:31.861896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:31.862551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:31.864255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:31.866188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:31.868403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 223/443] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:31.874852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:31.900377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:31.914476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:31.915957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.012743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4b12-1da1-41b9-b18a-7a3e2220ff92 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.032206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 221/444] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:32.033081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.053677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.054314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.054589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.054864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.056019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.058056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.059660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 224/445] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:31 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:32.060429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.097780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.147903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.159792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b395f-e58d-432f-91f5-ef9163e919c4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.184778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8161eb9-09a9-4b37-9f8a-8d0b74e642f4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.205358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.206265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.206757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.207240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.219355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:32.223482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:32.224003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:32.224890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:17:32.225748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:32.226355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.226355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.226355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] PATH_INFO: `/v3/projects/0d1859c09d264db597aab109f2d67e77` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.226534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 222/446] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:32.226597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] RBAC: Authorizing `identity:get_project(project_id=0d1859c09d264db597aab109f2d67e77)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.226669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.227629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.229648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 225/447] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/projects/0d1859c09d264db597aab109f2d67e77 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:32.232574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.345588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.348869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5707-e83a-400c-8aba-5cac07bed9ed None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.351904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd95b9-a5c5-4df8-a14f-27a794ee613d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.387606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.388205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.388477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.388743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.389430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:32.389721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.391185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.392432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.393142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.393811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.394087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.394923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=fed56dd461254bc682cf1955076ea721, scope.project.id=0d1859c09d264db597aab109f2d67e77) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:32.395229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.396668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.408290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 226/448] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:32.408290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.408632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 223/449] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=fed56dd461254bc682cf1955076ea721 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:32.409675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.440139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.536166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654479be-8ff2-47b4-a230-cd60443fa1fe None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.542460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f686f6-65f0-4b54-ab63-edd6455c9ca4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.575673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.576311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.576599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.576874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] PATH_INFO: `/v3/groups/e49e2afcb344451bb1ce073aee63a862` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.578028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] RBAC: Authorizing `identity:get_group(group_id=e49e2afcb344451bb1ce073aee63a862)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.579424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.580017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.580624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.582210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.582210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.587052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 227/450] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/groups/e49e2afcb344451bb1ce073aee63a862 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:32.587932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.627529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.669586 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.672263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.698921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.715431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c64e-c5d7-468d-81e4-c438a9386378 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.758039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.758653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.758931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.759203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.760571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] RBAC: Adding query filter params (group.id=e49e2afcb344451bb1ce073aee63a862, role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=0d1859c09d264db597aab109f2d67e77) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:32.761052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.762476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.768603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 228/451] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/role_assignments?group.id=e49e2afcb344451bb1ce073aee63a862&scope.project.id=0d1859c09d264db597aab109f2d67e77&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:32.799686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 224/452] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:32.805886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:32.917901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:32.925199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919172ca-8022-4948-a645-d604f1dbbf8e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:32.963900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:32.964522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:32.964842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:32.965103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] PATH_INFO: `/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:32.966264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] RBAC: Authorizing `identity:get_project(project_id=35c28eab9a294b9ba17b7f5b9c24d868)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:32.967696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:32.969519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 229/453] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/projects/35c28eab9a294b9ba17b7f5b9c24d868 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:32.975093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:33.081102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc58a-d285-49cc-808c-9dbaf20c9d66 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:33.100848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:33.118790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697f39f-177d-491a-bae5-33d68502e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:33.119508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f39f-177d-491a-bae5-33d68502e287 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:33.119780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f39f-177d-491a-bae5-33d68502e287 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:33.120043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f39f-177d-491a-bae5-33d68502e287 None admin] PATH_INFO: `/v3/groups/f8d794d38a9b47d8a91f3a37ccda8548` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:33.121165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697f39f-177d-491a-bae5-33d68502e287 None admin] RBAC: Authorizing `identity:get_group(group_id=f8d794d38a9b47d8a91f3a37ccda8548)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:33.122595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697f39f-177d-491a-bae5-33d68502e287 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:33.124310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 225/454] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Aug 24 03:17:32 2022] GET /identity/v3/groups/f8d794d38a9b47d8a91f3a37ccda8548 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:33.130210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:33.232799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce03d-a42f-4b3a-84ec-1c1564d1fc99 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:33.266304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:33.266895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:33.267172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:33.267444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:33.268159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] RBAC: Adding query filter params (group.id=f8d794d38a9b47d8a91f3a37ccda8548, role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=35c28eab9a294b9ba17b7f5b9c24d868) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:33.268464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:33.269927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa214f-4748-46b0-9919-a4aa8f41fdd0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:33.276406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 230/455] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Aug 24 03:17:33 2022] GET /identity/v3/role_assignments?group.id=f8d794d38a9b47d8a91f3a37ccda8548&scope.project.id=35c28eab9a294b9ba17b7f5b9c24d868&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:35.707659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34b87-6ee4-4c3a-8020-da057286ff5a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:35.708200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34b87-6ee4-4c3a-8020-da057286ff5a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:35.708200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34b87-6ee4-4c3a-8020-da057286ff5a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:35.709857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 226/456] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:35.715660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7ed3a-f459-47ef-9452-71a4071b65c1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:35.716070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7ed3a-f459-47ef-9452-71a4071b65c1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:35.716237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7ed3a-f459-47ef-9452-71a4071b65c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:35.742047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-32e7ed3a-f459-47ef-9452-71a4071b65c1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:35.743539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-32e7ed3a-f459-47ef-9452-71a4071b65c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:35.774695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:35.834703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 231/457] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:35.862837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b5aa-27c7-4a86-85b2-a85d2b6d85bb None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:35.863481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b5aa-27c7-4a86-85b2-a85d2b6d85bb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:35.863955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036b5aa-27c7-4a86-85b2-a85d2b6d85bb None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:35.865839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 227/458] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:35.871112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:35.871399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:35.871669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:35.892856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:35.894062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:35.925678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3800e-625a-4e0b-af79-650d645f0898 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:35.926254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3800e-625a-4e0b-af79-650d645f0898 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:35.926742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3800e-625a-4e0b-af79-650d645f0898 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:35.928280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 228/459] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:35.934710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:35.935229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:35.935734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:35.968546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:35.970129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:36.014605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 232/460] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:36.059896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 229/461] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:36.145080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.247354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:36.262879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7ce42-ef57-40de-a66d-858bf53071ba None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.298252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.299053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.299330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.299651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.310965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.312408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:36.327516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:36.328605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:36.329703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:36.329703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:36.329703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:36.329703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 233/462] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:36.338375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.408089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.478201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0069-2e3a-4577-b4db-fc7ab1da79da None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.500866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:36.518511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.519116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.519392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.519761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.520504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:36.520847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.522235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:36.535966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59402fb-de46-4132-a57b-fa6ba7ef37e7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.542772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 230/463] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:36.543638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.582618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.583243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.583524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.583797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.591489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:36.595591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.597412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:36.612552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:36.615494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:36.616326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:36.616326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:36.616326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:36.616326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 234/464] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:36.616326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.658798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea20b21-7b66-4823-ad34-35999ed90302 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.695953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.696585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.696902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.697175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.708549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.710199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:36.724145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda8e6a-130f-489b-9eb7-0c1d6fe33e1c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1db868e1-f149-4c83-93cc-af3e95a6c5c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:36.726131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:36.727818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:36.728651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:36.728651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:36.728651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:36.728651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 231/465] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:36.729352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d636d-4d3c-4041-9bf2-58eb688a154c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.729431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d636d-4d3c-4041-9bf2-58eb688a154c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.729657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d636d-4d3c-4041-9bf2-58eb688a154c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.731068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 232/466] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:36.731576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d636d-4d3c-4041-9bf2-58eb688a154c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.760284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:36.772529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.773175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.774011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.774296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.774949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.776420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:36.799043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 235/467] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:36 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:36.800260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:36.863282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d636d-4d3c-4041-9bf2-58eb688a154c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.864789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:36.931203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.932050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.932780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.933219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.934573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:36.935009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.937121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d45ef-4ac4-4ade-9ee8-34879659a75d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:36.943965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3700622-e5ec-4db3-b675-38cc7c7faa62 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:36.954281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 233/468] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:36.956306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.956671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.957041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.988812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:36.989984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:36.993560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:36.994475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:36.994475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:36.994475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:36.994978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:36.995275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:36.997931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:37.010590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 236/469] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:37.011685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:37.076117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 234/470] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:37.077299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:37.099652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:37.142417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd4ef5-b7be-4795-a265-1baaa57d5f56 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:37.153240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:37.191613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068c5f7-af6b-4a4f-87fb-89720536f30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:37.192647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068c5f7-af6b-4a4f-87fb-89720536f30c None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:37.193096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068c5f7-af6b-4a4f-87fb-89720536f30c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:37.193836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068c5f7-af6b-4a4f-87fb-89720536f30c None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:37.194916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068c5f7-af6b-4a4f-87fb-89720536f30c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:37.196910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068c5f7-af6b-4a4f-87fb-89720536f30c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:37.200126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff6689-8c90-420b-952f-2e721b55c71c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:37.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 237/471] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:37 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:37.253630 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167fa4d-9d64-4ca2-a308-06194fc5bb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:37.254363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167fa4d-9d64-4ca2-a308-06194fc5bb62 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:37.254685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167fa4d-9d64-4ca2-a308-06194fc5bb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:37.254963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167fa4d-9d64-4ca2-a308-06194fc5bb62 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:37.255656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167fa4d-9d64-4ca2-a308-06194fc5bb62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:37.257172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167fa4d-9d64-4ca2-a308-06194fc5bb62 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:37.286953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 235/472] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:37 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:37.308355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588af3e-c60a-439b-b479-c04cef297232 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:37.308680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588af3e-c60a-439b-b479-c04cef297232 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:37.308944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588af3e-c60a-439b-b479-c04cef297232 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:37.310727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 238/473] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:37.318996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:37.319316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:37.319542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:37.350082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:37.351430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:37.457254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 236/474] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:37.563716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588af3e-c60a-439b-b479-c04cef297232 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:37.700545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588af3e-c60a-439b-b479-c04cef297232 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:37.743673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:37.744324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:37.744553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:37.744833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:37.759357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:37.760858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:37.763152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:37.768791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:37.778986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:37.779974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:37.780840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:37.780840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:37.780840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:37.783014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 239/475] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:37.791586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:37.806021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:37.913238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8d779-9b1e-4f90-a193-b8a1834fce38 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:37.957933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:37.958706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:37.959078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:37.959441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:37.960211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:37.960711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:37.962596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:37.976209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 240/476] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:37.981900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:38.030395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43479a54-c26e-4e2a-ab36-ef74b162634f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:38.068024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:38.105347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:38.107165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:38.108086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:38.109131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:38.110860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:38.113658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:38.122328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b15753-1321-46f3-a27a-11337254b81e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:38.128694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.sql.core [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:17:38.128694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 03:17:38.128694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [parameters: {'id': '2527b0a0273044c8aa8f0ff482ec5f7f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 03:17:38.128694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81485) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Aug 24 03:17:38.165547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:17:38.166623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application yield resource Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise self.value Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application session.commit() Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:17:38.167764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:17:38.168687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:17:38.169794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:17:38.170813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [parameters: {'id': '2527b0a0273044c8aa8f0ff482ec5f7f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:38.171713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 03:17:38.172693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:38.181523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 237/477] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Aug 24 03:17:37 2022] POST /identity/v3/roles => generated 144 bytes in 414 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:38.187771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:38.213381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:38.214291 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:38.214714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:38.215160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:38.216025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:38.218834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:38.267488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 241/478] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:37 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:17:38.305400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:38.349408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef0d2-62a7-4047-9d2e-5d5975282116 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:38.388942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee4d65-4401-40e3-9ba7-40e266bcb737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:38.389582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4d65-4401-40e3-9ba7-40e266bcb737 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:38.389860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4d65-4401-40e3-9ba7-40e266bcb737 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:38.390133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee4d65-4401-40e3-9ba7-40e266bcb737 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:38.403078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee4d65-4401-40e3-9ba7-40e266bcb737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1cee4d65-4401-40e3-9ba7-40e266bcb737 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:38.408515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:38.409575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:38.410657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 238/479] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:17:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:38.416401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:38.545501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461303f-4dea-4319-ab72-9c98cee59285 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:38.576739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:38.584021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:38.584714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:38.585145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:38.586185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:38.587031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:38.587912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:38.592338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41e901-27b0-4497-86bc-44acf7891282 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:38.603324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 242/480] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:39.111778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617ac5-bbdf-4698-80c7-8d7bd2527634 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:39.112282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617ac5-bbdf-4698-80c7-8d7bd2527634 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:39.112505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617ac5-bbdf-4698-80c7-8d7bd2527634 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:39.114355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 239/481] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:39.119964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5941b03-5ba0-41db-8193-d09c4a496f1a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:39.120488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5941b03-5ba0-41db-8193-d09c4a496f1a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:39.120646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5941b03-5ba0-41db-8193-d09c4a496f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:39.145138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a5941b03-5ba0-41db-8193-d09c4a496f1a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:39.146369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a5941b03-5ba0-41db-8193-d09c4a496f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:39.234780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 243/482] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:39.615861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2e55c-4fce-4384-b751-009ff3397f9a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:39.617242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2e55c-4fce-4384-b751-009ff3397f9a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:39.617731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2e55c-4fce-4384-b751-009ff3397f9a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:39.619553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 240/483] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:39.625058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:39.625648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:39.626143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:39.649221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:39.650768 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:39.733208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 244/484] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:39.761297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2e55c-4fce-4384-b751-009ff3397f9a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:39.906555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2e55c-4fce-4384-b751-009ff3397f9a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:39.946399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:39.956597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fd744-42ec-46b6-a082-3a64585e9a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:39.957249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fd744-42ec-46b6-a082-3a64585e9a3c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:39.957523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fd744-42ec-46b6-a082-3a64585e9a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:39.957787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fd744-42ec-46b6-a082-3a64585e9a3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:39.978785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fd744-42ec-46b6-a082-3a64585e9a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e34fd744-42ec-46b6-a082-3a64585e9a3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:39.983695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:39.984594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:39.985899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 241/485] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:39.994036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.022252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13cf16-84ea-44d2-a145-5795dd04563a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.022628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13cf16-84ea-44d2-a145-5795dd04563a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.022913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13cf16-84ea-44d2-a145-5795dd04563a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.024243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 242/486] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:40.038902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.039174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.039444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.067587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:40.068467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:40.070306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:40.118846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903dc54-94ee-45f6-99bd-f31fcaa1df9c None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.156693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.157478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.157673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.158015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.158643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 243/487] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:40.159055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:40.159187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:40.160125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.160695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:40.172009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 245/488] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:40.177815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.357798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff20cc-7bd0-4632-9d00-1760028ca149 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.373940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ba592-c721-4995-a0f2-31cfb3106f1f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.391640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:40.403814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:40.432705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.433276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.433549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.433821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.435931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.437795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.438090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.438355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.453279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:40.456099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:40.467342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:40.472004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:40.472893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:40.473736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:40.475230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 244/489] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:40.480664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:40.485839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:40.487682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:40.488822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:40.488822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:40.488822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:40.488822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 246/490] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:40.488822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.611030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56694ae-0d6d-4379-8c09-4f05826b7d38 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.619382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369a344-89a4-4ca0-a43b-e48d1e9a1d25 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.660739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.661370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.661672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.661978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.663098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:40.663407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:40.664795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:40.665337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.666063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.666558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.666899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.674627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 245/491] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:40.675525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.685989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:40.692961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:40.694173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:40.695383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:40.697333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 247/492] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:40.698131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.727969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:40.737657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:40.804427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621248c-818c-4087-89ef-3da58db52349 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.824296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c03e25-f1ad-4e29-bf60-63de32373dcf None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.848582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.849229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.849543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.849780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.850557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:40.850867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:40.852478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:40.860402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:40.861491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.861784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.862051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.864494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 246/493] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:40.865151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.877875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:40.879331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-a1399204-c89a-4d84-8bcd-9249ec07da94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:40.897622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:40.898653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:40.899470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:40.899470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:40.899470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:40.903472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 248/494] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:40.904666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b951-81dc-4c2e-baaa-ced5f4d2c441 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:40.904975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b951-81dc-4c2e-baaa-ced5f4d2c441 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:40.905269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b951-81dc-4c2e-baaa-ced5f4d2c441 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:40.908295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 249/495] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:40.908295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b951-81dc-4c2e-baaa-ced5f4d2c441 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:40.978311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:40.984118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a7d1e-615e-4fa3-84a9-e91ce1d17d0b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:40.997561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.021485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b951-81dc-4c2e-baaa-ced5f4d2c441 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.023064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.023670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.023946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.024218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.024912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:41.025216 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.026601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.039472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 247/496] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:41.040351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.058962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.059586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.059862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.060130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.067956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.070557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-13da7a93-7bf9-4a69-8677-7bd2fd587a3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:41.082899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:41.083868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:41.084828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 250/497] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:17:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:41.085489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.085558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.086090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.103067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:41.104283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:41.146428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6bfe5-f980-4c60-a6ab-5ead95195fc1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.172461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 251/498] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:41.173412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.188407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.189166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.189451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.189718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.190350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:41.190652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.192002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c92d-d1b3-4ca1-a433-bae35553476d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.204493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 248/499] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:41.205690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06427f08-f27e-4ce6-a73f-b3ff0d189612 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.205966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06427f08-f27e-4ce6-a73f-b3ff0d189612 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.206234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06427f08-f27e-4ce6-a73f-b3ff0d189612 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.207661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 249/500] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:41.208353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06427f08-f27e-4ce6-a73f-b3ff0d189612 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.231209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.248037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.288336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8823fd-1cca-4ef0-a0b9-a6a7fb971b10 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.321031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06427f08-f27e-4ce6-a73f-b3ff0d189612 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.331632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.332255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.332531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.332819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.343457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.344968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.358388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.358988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.359264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.359531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.360249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:41.360549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:41.361822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:41.362673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:41.363506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:41.363506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:41.363506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:41.363667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fede14d-f915-4105-b515-97d5ae554bf0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.363742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 252/501] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:41.364069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.401436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 250/502] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:41.403036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.403494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.403962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.422949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:41.424138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:41.476126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e2549-6d1f-4085-bf1f-aa8b7818e821 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.494951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 251/503] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:41.495714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.506225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.513630 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.515705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.516534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.516839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.517114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.525569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.527013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:41.538529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:41.539437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 253/504] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:41.540358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.616730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0ec1-5b43-44a0-acdc-cc935460a611 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.649491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34c93f-ed4d-4a42-942d-532e1fdad793 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.652614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.653244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.653518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.653783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.654417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:41.654712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.656095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.672392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 252/505] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:41.672392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.694317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.694992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.695340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.695669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.741623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.748408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.770575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.772712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.778280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6db2c0-f608-4c48-81c9-066cf75b29b7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.812754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.813512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.813788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.814055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.824345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:41.829385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:41.830457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:41.831427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 253/506] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:41.832241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.875780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 254/507] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:41.876608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:41.934745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0e897-3aa7-4712-b589-242ecdf0faec None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.960455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.961373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:41.968412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:41.969029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:41.969304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:41.969578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:41.970291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:41.970589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:41.972006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:41.981924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0e8a5-c007-4217-bb29-bf4ac695c589 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:41.989318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 254/508] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:41.990108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.017011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.017556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.017828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.018099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.026203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.027658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:42.039824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:42.040758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:42.041655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 255/509] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:42.042209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.095387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258897d9-6ad2-425f-b1fb-88f66699aabb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.129392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.130007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.130283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.130555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.141656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.143102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.145525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479df8-40a8-468f-a410-e7e16125b0fe None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:42.158461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:42.159366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:42.160229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:42.160229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.160229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:42.160229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 255/510] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:42.161764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.182358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.182962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.183237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.183511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.184167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:42.184464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.185887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.194469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 256/511] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:42.195777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.199607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.208747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.272604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb08a8-b4e7-46f6-8a74-0178cbcf11a9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.303833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078767d-2fe8-4f5b-969a-e2246bb720fb None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.307844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.308471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.308751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.309082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.319480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.321051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:42.336448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:42.337414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:42.338247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:42.338247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.338247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:42.338413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.338413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.338548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 256/512] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:42.338612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.338612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.338734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.406556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.408091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.410011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.432689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.442685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ca81-af3d-4c8a-a31c-d7e34fc3bbd3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.476412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.477007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.477280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.477546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.478257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:42.478562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.479997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.497092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 257/513] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:42.497875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.510402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 257/514] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:42.511222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.597651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6d558-4024-44df-9cc1-8903f8e4bd38 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.610284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70798dd5-1a5a-42f8-803f-2153232c198b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.611912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.632464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.633097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.633375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.633645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.634273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:42.634576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.635950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.645481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.646095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.647025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.647504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.647627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 258/515] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:42.648340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.665083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.667206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.668697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:42.683234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:42.684144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:42.685555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:42.685555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.685555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:42.685555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 258/516] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:42.685555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.756306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdfc4-1ba5-4543-a584-93604a659ae0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.788399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701d866-d86a-416e-89ea-f84a4637f224 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.791028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.791625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.791902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.792171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.792824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:42.793127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.794530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.806874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 259/517] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:42.808084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:42.822852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.823444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.823716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.823981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.848575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.879510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:42.894431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.895917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.916262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a6072-705d-458f-9a20-0789b7c3bb71 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:42.950315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:42.950936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:42.951206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:42.951472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:42.961987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:42.963500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:42.978431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:42.979331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:42.980203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:42.980203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:42.980203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:42.980203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 260/518] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:42.980591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.000354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 259/519] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:43.001247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.056009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.087314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b30e-6a27-459f-82d3-3a110c48120d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.102822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde89cf9-0bf5-4862-9e3e-b00f65013d28 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.106896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.121092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.121691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.121967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.122237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.122873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.124306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.137546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.138221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.138603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.138951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.139661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:43.139969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.141409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.145260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 261/520] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:17:42 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:17:43.146068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.160490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 260/521] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:43.161422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.270615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768822e7-74b4-44e7-9b59-c3a74f830cf8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.279899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ffa1d-aa6f-421c-951c-be1a7a5c4437 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.296362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.317349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.317349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.317349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.317349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.324731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.325375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.325679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.326910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.327015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.327104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:43.327104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.328925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.329032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.342832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 261/522] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:43.343672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:43.343907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:43.344720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:43.345580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 262/523] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:43.346567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.388994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.456330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff57d1-d95e-4fba-a324-87068ecf43ae None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.460366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e1b55-7926-44f6-90e6-8425ce5d8f80 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.493561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.494187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.494472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.494728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.497774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.498398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.498677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.498951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.506125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.507187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.508193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.508635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:43.521808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:43.522859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:17:43.523721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:43.524219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:43.530398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:43.531234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:43.531234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:43.531234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:43.531234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 262/524] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:43.531234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.531494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 263/525] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:43.531494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.562649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.633560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.640478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b637462-8605-4b59-ad1c-345bff444076 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.647504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670c254-77c6-4719-b257-aed06a5c6d7a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.676555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.677187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.678022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.678358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.679049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:43.679354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.680798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.684207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.684861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.685145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.685419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.692324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.693911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.703678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 264/526] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:43.703678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:43.706190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:43.706992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:43.707814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 263/527] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:43.709029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.799243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.813296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214e68-9e3c-4308-80e2-f1bcf615823f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.820304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911add3-b98f-4ca7-b1e4-29653b3dc592 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:43.853923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.854627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.854939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.855157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.856343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:43.856746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.857630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:43.858230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:43.858321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.858500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:43.858770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:43.859406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:43.859712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:43.861088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:43.874050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 264/528] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:43.874859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.880753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 265/529] 10.222.0.81 () {56 vars in 1139 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:43.881802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:43.901659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:43.987058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcceac-0ecc-4af1-bbcd-4b40839a30ed None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.008742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0166f-4cae-49c5-9af6-50ef8246c358 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.023525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.024494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.025351 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.025805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.026598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=b8afb31c267946b7851cb1c22b62b6cb) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:44.026906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.028331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.034251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 265/530] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/role_assignments?user.id=b8afb31c267946b7851cb1c22b62b6cb&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:44.035108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.056858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:44.066123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.067649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.068412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.069359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.070880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:44.071482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.073527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.089592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 266/531] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:44.090841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.162584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d828cf-68b0-41f0-9424-8983556a1e90 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.199121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:44.205795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.206449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.206721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.207011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.218006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.219861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:44.234152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:44.235036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:44.235857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:44.236262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2ce9a-001a-43ef-b275-cdc3a813f428 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.238010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 266/532] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:44.239024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.279806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.281161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.281639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.282108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.332791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:44.361617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.363521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.374172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c3160-9aa1-454d-86d3-b4bbb9bf4242 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.419983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.420499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.420800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.421135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.421864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=c4bd5bc35c41458b9ba20d87738cde16) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:44.422168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.424487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.432724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 267/533] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/role_assignments?user.id=c4bd5bc35c41458b9ba20d87738cde16&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:44.433937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.456824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:44.504385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 267/534] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:44.524059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.569375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e5f1-3acf-416d-9f8a-bda519301198 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.609551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.610187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.610462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.610732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.611818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:44.612125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.613614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.619347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:44.643513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 268/535] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:44.644867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.734914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39d0f-64c9-4ef7-91dc-101b1373e61f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.806389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.807739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.808830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.810407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.813180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5b8f1-97f9-42e7-9a2a-ed74690b082d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:44.833345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.835250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:44.871471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:44.872412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:44.873282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:44.873282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:44.873282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:44.877411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 268/536] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:44.882508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:44.893160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:44.894084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:44.894552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:44.895012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:44.896666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=06578427361b421fa275befc898a7f72) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:44.897214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:44.898922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:44.909603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 269/537] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/role_assignments?user.id=06578427361b421fa275befc898a7f72&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:44.941155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:45.033678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae9d40-e685-418b-97af-bee93db00e29 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:45.071927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:45.072548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.072841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.073111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.073774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:45.074072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:45.075500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b1b12-9aab-413a-a3bb-af7c22052545 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:45.088597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 269/538] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:45.094292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:45.175228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802013e8-b030-4339-8102-b7ed9e3333d6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.175535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802013e8-b030-4339-8102-b7ed9e3333d6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.175785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802013e8-b030-4339-8102-b7ed9e3333d6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.178114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 270/539] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:45.179869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:45.185222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.185981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.186962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.223938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:45.225165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:45.247435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cba13-5b54-4ae9-88a1-1864a8d6280d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:45.305709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 271/540] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:45.310230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:45.311038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.311415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.311787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.319924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:45.321564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:45.334066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:45.335342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:45.336178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 270/541] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:45.341531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:45.369136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:45.450772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e4b10-8acb-4a8b-9f0b-2fa4236bd68d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:45.487115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:45.487715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.488044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.488319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.489071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:45.489704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:45.491162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:45.503115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 272/542] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:45.509224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:45.616378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b4d0-f455-4df0-aff6-159f9a4d40f2 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:45.641299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:45.656898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:45.658607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.658607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.658607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.659076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=12a2363821f445db980712d3e491e791) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:45.659076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:45.660538 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:45.671768 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 271/543] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:45 2022] GET /identity/v3/role_assignments?user.id=12a2363821f445db980712d3e491e791&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:45.727439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:45.822650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:45.847055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57d080-f56a-4a25-9196-82fe86514a9b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:45.885562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:45.886122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:45.886399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:45.886673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:45.904416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:45.914750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:45.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:45.916427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 273/544] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:45.920741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:46.030519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030a953-9326-48ff-8181-6b5926c1f52f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:46.065194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:46.065864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.066142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.066414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.067137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:46.067975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:46.069386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:46.077710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 272/545] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:46.090111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:46.241197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b70d5-1b75-4d59-bc29-b3ecd6c4f377 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:46.288778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:46.289414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.289727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.290000 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.293059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:46.302068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:46.310470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-915d7ef7-52ff-4feb-8994-80d9be6f30b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:46.326653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:46.327580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:46.329127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:46.329127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:46.329127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:46.329127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 274/546] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:46.333521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:46.425914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:46.493876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a02c-7953-419e-b9e1-d7031a752086 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:46.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562cc4-c5ef-4d1f-afce-00febe847fd4 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562cc4-c5ef-4d1f-afce-00febe847fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562cc4-c5ef-4d1f-afce-00febe847fd4 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 275/547] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:46.523432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375557-36d2-432c-8042-c038cd3dd59a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.523709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375557-36d2-432c-8042-c038cd3dd59a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.523974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375557-36d2-432c-8042-c038cd3dd59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.549946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:46.550875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.551374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.551944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.552840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:46.553340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:46.554923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:46.556036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-27375557-36d2-432c-8042-c038cd3dd59a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:46.557473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-27375557-36d2-432c-8042-c038cd3dd59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:46.579150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 273/548] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:46.588457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:46.645122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 276/549] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:46.734134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a796a8c-3755-4149-b8d1-9eb0f79a41a1 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:46.739859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:46.784292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:46.784864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.785151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.785417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.793469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:46.794914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:46.806557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e9dbd-7bae-4e0a-9623-41510ba65ac0 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.807136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e9dbd-7bae-4e0a-9623-41510ba65ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.807629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e9dbd-7bae-4e0a-9623-41510ba65ac0 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.809539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 277/550] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:46.814754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:46.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:46.817783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 274/551] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:46.819126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:46.819636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:46.820102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:46.828426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:46.853870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:46.855336 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:46.964643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 278/552] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:17:46.997764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.014653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676b8f-facd-46da-b2ec-d1186940bf30 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.057738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.090126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.091008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.091436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.091845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.096829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:47.097169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:47.098819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:47.118939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d2022-c4bb-49e4-b24d-2397c8f1c59c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.170627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.171676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.172149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.172615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.192497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 275/553] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Aug 24 03:17:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:47.193683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:47.198514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:47.200829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:47.201884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:47.202698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:47.212080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 279/554] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:47.226573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.258743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.353044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4543e-1ddb-4c12-921d-287f0fa86c6b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.368739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41db58-a0ea-4d3f-9b4e-a06ca4682600 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.391566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.392528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.393070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.393561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.401143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.426436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.427077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.427376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.427667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.428602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:47.428939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:47.431077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:47.442760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 280/555] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:47.443948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.477874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:47.479340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:47.503541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.557266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975dff5-bbec-4ba7-b76e-86d665c7ac73 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.591140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 276/556] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:17:47.591140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.599944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af85edf-c077-4043-babb-7d06cde48b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.600575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85edf-c077-4043-babb-7d06cde48b1f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.601009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85edf-c077-4043-babb-7d06cde48b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.601284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85edf-c077-4043-babb-7d06cde48b1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.612561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af85edf-c077-4043-babb-7d06cde48b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4af85edf-c077-4043-babb-7d06cde48b1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:47.618468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:47.619434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:47.620429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 281/557] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:47.621966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c702bec-d1b3-4a08-a1a2-0dac761ad4ee None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.622245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c702bec-d1b3-4a08-a1a2-0dac761ad4ee None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.622517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c702bec-d1b3-4a08-a1a2-0dac761ad4ee None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.628047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 282/558] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:47.628047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c702bec-d1b3-4a08-a1a2-0dac761ad4ee None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.659107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.704068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003447-4b02-4bee-aa09-0fdee303f5af None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.728949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.742809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.743522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.743861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.744195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.754944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:47.755892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c702bec-d1b3-4a08-a1a2-0dac761ad4ee None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.756499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:47.772195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:47.773395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:47.774292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:47.774292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:47.774292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:47.774292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 277/559] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:47.774514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.799706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.800334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.800611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.800922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.811946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:47.813461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1a8d7a70-1173-4154-bc9e-ec8453f19a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:47.828113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:47.829117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:47.830067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:47.830067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:47.830067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:47.830067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 283/560] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:47.830840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.830840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.830975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.850263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:47.851469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:47.888519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbf04-498b-416b-aaaf-ad1c998906ef None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:47.924586 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 284/561] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:47.925645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:47.936731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:47.937574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:47.937997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:47.938464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:47.939269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:47.939712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:47.941280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:47.948204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:47.952375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 278/562] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:47.953886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.006732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.052736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc3bf4-15fa-4aba-a8e3-3980d796d9a2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.077309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65b82-ac03-476a-b15d-1855c12a05f6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.092126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.092913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.093196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.093468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.094106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:48.094408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.095780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.108120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 285/563] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:48.109197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.118421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.119082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.119363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.119638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.120281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:48.120640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.122060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a9d63-225c-4ca2-8189-0ff0ed0656a2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.134428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 279/564] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:48.135673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3f5f-8226-4e4a-b64f-c241d3770f1a None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.135959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3f5f-8226-4e4a-b64f-c241d3770f1a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.136233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3f5f-8226-4e4a-b64f-c241d3770f1a None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.140928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 280/565] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:48.140928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc3f5f-8226-4e4a-b64f-c241d3770f1a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.188063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.220083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6f94a-6d88-4efd-a578-a8e5e3d9cf83 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.245833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.253201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc3f5f-8226-4e4a-b64f-c241d3770f1a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.255511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.256145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.256419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.256690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.267576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.268982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:48.283448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:48.284379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:48.285317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:48.285317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:48.285317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:48.285317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 286/566] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:48.285767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.291673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.292325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.292599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.292919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.300128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.301686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-e77c3e26-ece9-4900-91b7-ae1241bc845a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:48.313534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:48.314549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:48.315634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 281/567] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:48.316277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.316277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.316558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.335268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:48.336493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:48.392692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98675a0f-e03f-4638-a4c4-03b12cd5d30e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.403551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 282/568] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:48.404541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.418900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.432106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.432734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.433041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.433317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.440041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.441508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:48.453922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:48.455003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 287/569] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:48.456627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.499980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.516739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f801fa-8fae-4677-b4b8-ca8c2f178c7a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.552662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.553814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.554163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.554641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.567290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854c666-1b39-4239-8879-20ae4df71cc2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.567753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:48.572453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:48.573581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:48.574612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:48.575257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 283/570] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:48.575841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.610007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.610752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.611116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.611512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.612234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:48.612605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.614315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.626397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 288/571] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:48.631810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.679066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.686394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875b2e-6ce7-4eb7-9ece-93e0f8d3b652 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.721195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.721786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.722058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.722323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.723982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:48.724283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.726846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.737421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2af25-9136-4e05-b33d-e1b32833f3e0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.746176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 284/572] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:48.746879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.751448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.774552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.775133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.775409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.775680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.776364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:48.776667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.778134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.789929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 289/573] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:48.790791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.853214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274cbe1-df88-46b9-ba1f-4527ed671232 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.887347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.887940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.888656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.888953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.889298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b126c9-ea09-45d3-95e7-537cde3db5b8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:48.889620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:48.889917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.891289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.897741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:48.900575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 285/574] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:48.901491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.930625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:48.931179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:48.931449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:48.931715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:48.939075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:48.940518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:48.951918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:48.953058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:17:48.954218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:48.957785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 290/575] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:48.958610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:48.972665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.009622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a7d9-224d-4920-8cb7-9520dca6181f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.043193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.043808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.044081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.044349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.055742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:49.058338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f355e8c-081a-46ac-8d77-b6dad31b4b60 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:49.060726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:49.061799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:49.062802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 286/576] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:49.063638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.097275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.098875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.099161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.099430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.110006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.111458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:49.126134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:49.127209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:49.128124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:49.128124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.128124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:49.128124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 291/577] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:49.128124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.147774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.170655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abc189-a954-4627-b40e-08c8e5261923 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.197293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.205781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.206369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.206915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.208686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=9d8eb54303a94ec38dbb9209c71381da) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:49.209023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.210447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.215888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 287/578] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/role_assignments?user.id=9d8eb54303a94ec38dbb9209c71381da&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:49.216486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.244624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed18c29-bb0b-443f-a471-675856457a2f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.287474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.288277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.288657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.289061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.301105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.302912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:49.322556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:49.323789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:49.324949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:49.324949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.324949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:49.330140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 292/579] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:49.330140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.344507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b56a9-a5ee-4215-ac2a-43748777a4c9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.383527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.384335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.384673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.385043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.385845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:49.386212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.387928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.404703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.413225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 288/580] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:49.413225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.447824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e4722-d5df-4433-8cc5-d9a17695d3c8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.464806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.487265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.487979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.488364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.488710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.489463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:49.489837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.491335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.504904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 293/581] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:49.504904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.527883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19898-c7ed-42de-9d5f-de5ad21bb6e9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.568820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.569796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.570083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.570354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.571116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:49.571413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.572796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.591130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 289/582] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:49.592021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.609015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a6a97-3e18-4b4d-90e5-8875296455f5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.639550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.647501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.648114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.648393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.648665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.649383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:49.650145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.651555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.669288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 294/583] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:49.669288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.710910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906b38-f801-4fb4-bcc9-15424cbc0cea None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.721040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.748725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.749513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.749885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.750252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.762967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.764514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.779403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ef5e-ff20-4aac-a040-1fd2fcb0324c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:49.780622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:49.781797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:49.782714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:49.782714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.782714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:49.784097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 290/584] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:49.785221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.818627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.819424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.819793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.820155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.831600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.834014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:49.849714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:49.850889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:49.851976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:49.851976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:49.851976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:49.853019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 295/585] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:49.853877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.899900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd8f63-a832-433d-be52-f3b9c8f84717 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.905113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:49.941622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:49.942241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:49.942515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:49.942782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:49.949455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:49.950905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:49.962444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:49.963461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:49.964337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 291/586] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:49.964839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:49.975120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3fadb-559f-43cc-8912-55686b827420 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:49.991371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.012460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.013448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.013729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.013997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.023121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.024569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:50.036404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:50.037550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:17:50.038896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:50.042277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 296/587] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:50.042856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.074936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b0104-c143-4cc9-bfe0-5fbad14aa554 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.110707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.111308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.111586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.111857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.112500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:50.112826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.114759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.132112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 292/588] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:50.132112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.158667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cac1a-d9d8-4134-9a62-da2bff36eeb6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.160969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.198738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.199349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.199631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.199914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.200550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:50.200889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.202263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.219582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 297/589] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:50.219582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.227695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.249586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81ecf1-feda-4d93-80af-f0ca633b4e0f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.286020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.286677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.286956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.287260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.288142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:50.288478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.289996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.307471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 293/590] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:50.308589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.326100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0047c-4584-4336-8f9e-105665b0e69d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.370297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.370906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.371185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.371455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.372189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:50.372557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.373962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.392470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 298/591] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:50.393084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.419245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.423713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341c1ad-d85c-404b-83eb-b929fb0b4935 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.461427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.462178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.462646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.463014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.470210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.471789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.475462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:50.486219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:50.487122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:50.487991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:50.492488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 294/592] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:50.492926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.509528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d955a-2557-46f7-af90-cd4d1a0031a2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.549881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.550686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.551059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.551431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.559295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.561007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:50.573365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:50.574533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:17:50.575677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:50.576969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 299/593] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:50.582579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.602148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5d234-9eb6-493c-a98c-61d244d3dc10 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.637614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.638258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.638533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.638806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/b8afb31c267946b7851cb1c22b62b6cb/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.654174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=b8afb31c267946b7851cb1c22b62b6cb, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.655771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.673961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 295/594] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:50 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/b8afb31c267946b7851cb1c22b62b6cb/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:50.674135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.675379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.720040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f43de-600b-4a96-ad68-9c84d34842b6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.738013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.769937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.771121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.771561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.771963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.784779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.787084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:50.804493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:50.805670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:50.806729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:50.806729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:50.806729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:50.812688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294547e-f23d-4540-be75-c52ae8f41a07 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.813316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 300/595] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:50.814417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.851642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:50.852353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:50.852632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:50.852982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:50.853740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:50.854303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:50.855792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:50.869111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 296/596] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:50.869847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:50.963717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62a34-c3bf-469b-93ac-ad3c1089b09c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:50.997589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:50.999313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cb066-51ba-431c-8a2e-f3835bb82ce7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.004505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.005143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.005421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.005693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.006434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:51.006743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.008226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.014676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.033873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 301/597] 10.222.0.81 () {56 vars in 1139 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:51.034508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.052197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.053338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.053824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.054529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.055723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:51.056194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.058424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 297/598] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:51.081437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.145349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29452ca-db29-4e99-89a2-941636065234 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.182651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.183307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.183585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.183858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/c4bd5bc35c41458b9ba20d87738cde16/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.200462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c23283-9a38-4d40-8243-29260943e3f8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.200634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=c4bd5bc35c41458b9ba20d87738cde16, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.202239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.216728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 302/599] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:51 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/c4bd5bc35c41458b9ba20d87738cde16/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:51.217550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.250715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.251774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.252300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.252814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.267582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.269616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.275390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-948555fa-d7a6-4fc5-9432-64b25d81ed61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:51.291701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:51.292730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:51.293655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:51.293655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:51.293655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:51.296087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 298/600] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:51.297789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daef70-9fef-4e89-bd75-5c1b720d98ae None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.298302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daef70-9fef-4e89-bd75-5c1b720d98ae None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.298978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daef70-9fef-4e89-bd75-5c1b720d98ae None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.301439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 299/601] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:51.303688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daef70-9fef-4e89-bd75-5c1b720d98ae None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.335040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.359179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230b54-096c-4da2-a9a6-36e718fd02f8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.405848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.406853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.407331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.407810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.416312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.417981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0b0d2c90-8333-4b3e-90eb-dd01e6c86859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:51.430397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:51.431954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:51.432979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:51.434883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 303/602] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:51.436355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.436808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.437215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.438014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daef70-9fef-4e89-bd75-5c1b720d98ae None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.456386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:51.458484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:51.478387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.479687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.480056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.480420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.481265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:51.481664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.483558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.504507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 300/603] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:51.504507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.538927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 304/604] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:51.540001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.572990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.591679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.611493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e82f6b-2eee-4abf-a3dd-5389f35e0954 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.649355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.650343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.650816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.651277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.652209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:51.652793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.654297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35564f-ce2a-4bbf-971c-462108cc7c9b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.654650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.668096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 301/605] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:51.673444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.730441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.731068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.731352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.731623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.753141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.754874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:51.775293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:51.776240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:51.777141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:51.782359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 305/606] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:51.793815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.812353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459e41a-b8b9-4b8c-8e7d-c86414eba8bc None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:51.874331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:51.875574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:51.877559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:51.878297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/06578427361b421fa275befc898a7f72/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:51.909140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.912986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=06578427361b421fa275befc898a7f72, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:51.915173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:51.934785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 302/607] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:51 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/06578427361b421fa275befc898a7f72/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:17:51.936081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:51.949406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:51.965644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aafb0-633c-4fe6-bd46-f2d1696a9892 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.005999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.006971 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.007443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.007949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.008950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:52.009459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:52.011197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:52.024875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 306/608] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:52.031229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:52.084995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa8c2-bf6a-4827-af0a-0a25208fca38 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.135136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.135813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.136096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.136515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.149567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:52.154831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:52.155844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:52.156699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 303/609] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:52.162911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe590740-6903-48ea-8088-b3af75faded3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.163105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:52.200359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.201448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.201935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.202398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/12a2363821f445db980712d3e491e791/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.219433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=12a2363821f445db980712d3e491e791, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:52.221342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:52.226203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:52.233672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:52.240067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 307/610] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:52 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/12a2363821f445db980712d3e491e791/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:52.287775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f0f93-250d-4878-b745-df375a91c060 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.328219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.328875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.329159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.329440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.330125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:52.330428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:52.332422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:52.342310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 304/611] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:52.348665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:52.483575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cded05-750a-4c82-a872-9d700b99f35a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.522640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.523279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.523556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.523826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.535656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:52.537352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:52.555227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:52.556164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:52.557066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:52.557066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:52.557066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:52.557066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 308/612] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:52.562025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:52.673432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5db45-726f-4c96-b9e9-3a942d3af976 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.689634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:52.712318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.712965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.713243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.713518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.714177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:52.714480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:52.715873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a2b40-b7ef-43ad-88b4-6f83c3a25d9c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:52.733087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 305/613] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:52.734501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:52.792568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:52.845387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e819-5007-44f5-a26f-24bc38f03d49 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:52.882255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:52.882887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.883167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.883438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.891485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:52.892970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:52.903315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aab14-c253-4cde-a460-59df876edbb0 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.903598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aab14-c253-4cde-a460-59df876edbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.903879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aab14-c253-4cde-a460-59df876edbb0 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.905272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 306/614] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-30146082-c331-4d68-b42f-11e7471924f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:52.908796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:52.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:52.911393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 309/615] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:52.916245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aab14-c253-4cde-a460-59df876edbb0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:52.929829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:52.930112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:52.930389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:52.958288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:52.959751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:53.042653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 310/616] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:53.087322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aab14-c253-4cde-a460-59df876edbb0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:53.123285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:53.124077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.124432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.124789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.125599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:53.125979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:53.127483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:53.146439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 307/617] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Aug 24 03:17:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:53.151842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:53.198417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:53.259972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21eb417-167a-4294-8942-bce1e159ef5c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:53.296525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:53.297181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.297459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.297756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.308840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:53.310380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:53.325612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:53.326719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:53.327665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:53.327665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:53.327665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:53.327665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 311/618] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:53.332634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:53.394705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:53.440229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809655b-9604-4f34-927e-0aaa00d02e3e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:53.483677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:53.484289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.484569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.484866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.485522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:53.485825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:53.487195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:53.499451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 308/619] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:53.505738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:53.645699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:53.663957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa06070-dc16-4010-8ec4-9725c8590ba7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:53.668339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:53.700394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:53.701043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.701314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.701582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.708395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:53.709940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-90d64cd6-312a-4eb4-bf7b-3ecb90926795 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:53.721085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:53.722248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:53.723315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 312/620] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:17:53.724279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7103e24-09af-4970-9403-b14b43fb4fc8 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.724279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7103e24-09af-4970-9403-b14b43fb4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.724533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7103e24-09af-4970-9403-b14b43fb4fc8 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.730447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 313/621] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:53.731434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7103e24-09af-4970-9403-b14b43fb4fc8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:53.773098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:53.779795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62fee4-9a1a-411b-a337-38de15087c3b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:53.819371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e606b-b5e5-4e09-a273-aff5adae62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:53.820793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e606b-b5e5-4e09-a273-aff5adae62f6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.821364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e606b-b5e5-4e09-a273-aff5adae62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.822564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e606b-b5e5-4e09-a273-aff5adae62f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.835424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e606b-b5e5-4e09-a273-aff5adae62f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-098e606b-b5e5-4e09-a273-aff5adae62f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:53.841848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:53.842959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:53.844000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:53.847682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 309/622] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:53.849695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.850274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.850797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.861288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7103e24-09af-4970-9403-b14b43fb4fc8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:53.872190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:53.874317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:53.907662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:53.907662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:53.907662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:53.907662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:53.908101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:53.908101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:53.910065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:53.930674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 314/623] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:53.931778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:53.950587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:53.980898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 310/624] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:53.982081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.055049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cdd8f-671d-4b60-afc1-85ee3dcc7996 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.095897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.096655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.097156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.100820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.100820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:54.100820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:54.100820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:54.110688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 315/625] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:54.116183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.168113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:54.211322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:54.218814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b30b8-0551-4cad-9f39-1b74d8cf394c None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.238867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d459e7-e238-4556-9138-1d085d237614 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.277964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.278600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.278878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.279152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.292326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:54.294278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:54.310235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:54.311286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:54.312176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:54.312176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:54.312176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:54.312176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 316/626] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:54.317525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.317788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.318604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.318882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.319148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/9d8eb54303a94ec38dbb9209c71381da/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.350518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=9d8eb54303a94ec38dbb9209c71381da, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:54.351973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:54.384954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 311/627] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:17:53 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/9d8eb54303a94ec38dbb9209c71381da/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:17:54.421570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.457713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d227728-0264-4afd-b585-52e61ac19747 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.506686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.507355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.507571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.507853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.509369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:54.509683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:54.510209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:54.512323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:54.530232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 317/628] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:17:54.537029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.561481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528b4b1-816e-4547-9f5d-fc5ca12e4716 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.596280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:54.604346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd042f0-2e0e-4e5e-8b04-54716ffb4691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.605376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd042f0-2e0e-4e5e-8b04-54716ffb4691 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.605843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd042f0-2e0e-4e5e-8b04-54716ffb4691 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.606297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd042f0-2e0e-4e5e-8b04-54716ffb4691 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.621208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd042f0-2e0e-4e5e-8b04-54716ffb4691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-bbd042f0-2e0e-4e5e-8b04-54716ffb4691 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:54.626522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:54.627498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:54.628473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:54.632853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 312/629] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:54.634553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dca2bd-5a8b-4c87-80ea-186ff7385f0e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.635050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dca2bd-5a8b-4c87-80ea-186ff7385f0e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.635426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dca2bd-5a8b-4c87-80ea-186ff7385f0e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.636885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 313/630] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:54.637989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3157-5cf2-41a5-b359-2ab17121f550 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.638328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3157-5cf2-41a5-b359-2ab17121f550 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.638601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3157-5cf2-41a5-b359-2ab17121f550 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.640367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 314/631] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:54.641112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3157-5cf2-41a5-b359-2ab17121f550 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.679574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741580-9228-4fd3-ae85-5918e36ef460 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.714191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.714805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.715082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.715355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.723793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:54.725284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1ef72e22-520a-435c-8b88-c6a64b6aae75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:54.736992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:54.737982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:54.738918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 318/632] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:54.740039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.740039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.740039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.757587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3157-5cf2-41a5-b359-2ab17121f550 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:54.759244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:54.760529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:54.796437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:54.797167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.797441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.797711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.798379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:54.798657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:54.800198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafd11f-725b-40d3-89d2-d1a86bcca76b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:54.807433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:54.813384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 315/633] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:54.814696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:54.814982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:54.815253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:54.836054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:54.838281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 319/634] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:54.838650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.839508 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:54.908046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:54.933224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 316/635] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:54.934120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:54.965040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b718-a18d-4ba4-ae4a-c01cbea98f3e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.001463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.002224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.002598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.003479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.004438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:55.004827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.006342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.027425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 320/636] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:55.033329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.072953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.141232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401db4-8fa9-4a73-85a4-d0299a8688a2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.163904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb225360-29ec-412b-b515-3c770eeb0db4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.184719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769986a-c645-4412-9020-09c042250d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.185771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769986a-c645-4412-9020-09c042250d91 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.186243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769986a-c645-4412-9020-09c042250d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.186725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769986a-c645-4412-9020-09c042250d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.198617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769986a-c645-4412-9020-09c042250d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.200410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769986a-c645-4412-9020-09c042250d91 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.212716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.213386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.213665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.213941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-4769986a-c645-4412-9020-09c042250d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:55.221428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:55.223203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:55.224737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:55.224737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:55.224737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:55.227865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 317/637] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:55.229766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769986a-c645-4412-9020-09c042250d91 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.274162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.297512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.299924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.338474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.342963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769986a-c645-4412-9020-09c042250d91 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.380025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.380984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.381492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.381954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.392466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:55.398320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:55.399246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:55.400089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 318/638] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:55.401136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.413969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 321/639] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:17:55.414824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.504666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.512990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c649a-ee3e-41be-8bcc-8e260a50d9be None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.535390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a300-1735-4c4c-9154-1ecf99cd6f49 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.562277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.563239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.563692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.564088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.565264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:55.566478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.568896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.593964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 319/640] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:55.593964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.595613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.597261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.597740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.598195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.604126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.619201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:55.629462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:55.630543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:55.632112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 322/641] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:55.638189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.733144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72a4f6-d218-4cc9-9072-6a55f938e0ea None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.759738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605ea74-c091-4b0a-bb08-51d5d11ac997 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.769986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22970835-993b-4882-b794-f6a296a0200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.770651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22970835-993b-4882-b794-f6a296a0200c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.770990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22970835-993b-4882-b794-f6a296a0200c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.771328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22970835-993b-4882-b794-f6a296a0200c None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.772051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22970835-993b-4882-b794-f6a296a0200c None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:55.772410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22970835-993b-4882-b794-f6a296a0200c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.773877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22970835-993b-4882-b794-f6a296a0200c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.786975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 320/642] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:55.786975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22970835-993b-4882-b794-f6a296a0200c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.802721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.803570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.804137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.805063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.805415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.819291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.820879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:55.836974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:55.837993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:55.843734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:55.843734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:55.843734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:55.843734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 323/643] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:55.843734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:55.897433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22970835-993b-4882-b794-f6a296a0200c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.899313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:55.937405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:55.938028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:55.938304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:55.938573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:55.948034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:55.949507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:55.959544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47a23b-38a3-4e18-982c-5337154f378a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:55.960544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:55.961678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 321/644] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:55.962757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.006400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.007356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.007627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.007891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.008567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:56.008860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.010288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.023534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 324/645] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:56.023534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.052142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.075876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd980e-9e33-4a1b-9eb8-6bf76c982012 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.112645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.113304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.113576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.113841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.125185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.126685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.129578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521ed5-64cb-4608-8fee-505e221b9de4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.140529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:56.141777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:56.142724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:56.144241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:56.144241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:56.144241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:56.144857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 322/646] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:56.146912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.175988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.176650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.176970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.177239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.177871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:56.178166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.179529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.192478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 325/647] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:56.193318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.255693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876a1e-0ed0-4dab-bb5c-07f1fb025bfa None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.287503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.294188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.294775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.295058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.295329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.296055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:56.296366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.297853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.302298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44baa0-3fc6-44d7-bc6e-12d7c0aa4e71 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.316603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 323/648] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:56.317478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.343583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.344195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.344469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.344755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.356156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.357628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:56.373049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:56.374221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:56.375112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:56.375112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:56.375112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:56.375112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 326/649] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:56.375335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.402075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.437407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef79977-b98b-4dc8-9fd1-83f721738f4f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.475143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.475836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.476177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.476515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.477432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:56.477734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.479363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.484796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94beffb-cd8e-4e12-94a8-fae3196d5bb5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.497038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 324/650] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:56.497038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.521202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.521819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.522093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.522361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.528951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.530382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:56.541746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:56.543191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 327/651] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:56.544279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.551753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.603407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509d88e-dfdd-4b8f-b6d2-2485b9a58a67 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.627047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.637528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f70a9-7534-4e0f-88d2-195e7afc9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.638126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f70a9-7534-4e0f-88d2-195e7afc9c2e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.638405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f70a9-7534-4e0f-88d2-195e7afc9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.638676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f70a9-7534-4e0f-88d2-195e7afc9c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.646253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9320-57b9-4070-a183-9afb062a543c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.680482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.681110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.681388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.681661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.682299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:56.682601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.683982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.696616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 328/652] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:56.697438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.706253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f70a9-7534-4e0f-88d2-195e7afc9c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.707703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f70a9-7534-4e0f-88d2-195e7afc9c2e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.756695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.799971 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6f1b-6fed-45cc-b3ee-294ed4166774 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.810398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 325/653] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:56.811712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343609f-526c-4f45-b2ec-dca504a75b26 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.812064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343609f-526c-4f45-b2ec-dca504a75b26 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.812406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343609f-526c-4f45-b2ec-dca504a75b26 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.813742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 326/654] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:56.814436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343609f-526c-4f45-b2ec-dca504a75b26 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.843440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.844060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.844333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.844600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.851956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.853694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:56.865669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:56.866583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 329/655] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:56.867869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.870673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:56.925740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343609f-526c-4f45-b2ec-dca504a75b26 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.959131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:56.959721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:56.959990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:56.960255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:56.960970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:56.961332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:56.962747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:56.971137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32c4b6-0983-4dc0-829a-e4bade67eb4a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:56.974740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 327/656] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:56.975548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:56.986973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.007430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.008048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.008320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.008588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.016164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.017659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-85366fbd-218b-4046-b1ef-75b7f935d392 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:57.029709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:57.030806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 330/657] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.032271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.033164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.050244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:57.051354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:57.084284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf6c7d-aebd-4551-b1c1-5488fefae9df None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.108157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.119406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.119919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 331/658] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:57.120100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.120375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.120694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.120816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.132119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.133608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:57.148396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:57.149575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:57.150458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:57.150458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:57.150458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:57.150458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 328/659] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:57.150680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.218190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.245061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e308fb-b146-4cda-934d-a951f0c290b0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.263732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2db1bd-23c5-4336-be08-34883193a10d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.282466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.283122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.283400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.283668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.284798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:57.285095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.286526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.298025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.298626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.298905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.299177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.299929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=b8afb31c267946b7851cb1c22b62b6cb) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:57.300240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.301740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.304048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 332/660] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:57.308820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.312985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 329/661] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/role_assignments?user.id=b8afb31c267946b7851cb1c22b62b6cb&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:57.313667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.391683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.438741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45dc84-13f2-46f7-af50-b911a9500fa7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.451102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44871e4-b572-4943-b131-6918f6183ff9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.470604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.485629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.486236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.486514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.486784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.487957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:57.488254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.489668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.492755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.493512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.493863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.494143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.494871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:57.495181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.496601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.505262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 330/662] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:57.505950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.519117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 333/663] 10.222.0.81 () {56 vars in 1139 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:57.520163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.630899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb515a64-c2c7-4e39-b0ab-1cf1af6ad4a9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.633357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8191-1910-4482-bd58-e9cbea66a2e5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.643973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.668671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.669412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.669623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.669958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.672300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.673065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.674101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.674101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.685395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:17:57.690499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:57.691520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:57.692360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 334/664] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:57.693647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.728348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.746009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.747475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.804217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d1446-31fb-4618-ba21-34c3532d709f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.840951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:57.841576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:57.841845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:57.842108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:57.849451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:57.850888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:57.858651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 331/665] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:57.859970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:57.862791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:57.863694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:57.864617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:57.868489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 335/666] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:57.869364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:57.903077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.966075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:57.981669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2e5d-f0bf-4668-8dc7-f93f2aad6f2f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:57.983186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd21b2-c5fe-4f2b-8925-87c5d3f93a1a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.019356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.019952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.020316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.020587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.021145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.021257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:58.021559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.021747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.022028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.022301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.023026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.031632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 336/667] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:58.032230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.098953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.100549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.126189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.136686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8720d0-d6b0-433a-821f-18071d7a7dd5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.172154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.172697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.173008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.173275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.184802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.186261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.194853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:58.201847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:58.202776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:58.203649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:58.203649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:58.203649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:58.203649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 337/668] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:58.203649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.225669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 332/669] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:58.226826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.312857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e48c5e-1a07-4c7c-8158-b23cde23a8da None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.338523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff78cd-c494-4b59-bbaf-c8705878afc8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.351120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.351730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.352008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.352279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.353254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:58.353558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.354987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.367686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 338/670] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:58.368494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.371065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.381380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.382004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.382285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.382558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.393769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.395347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:58.411894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:58.413082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:58.413957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:58.413957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:58.413957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:58.413957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 333/671] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:58.413957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.455952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.488196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7815e9-2218-4997-8457-b955ba3860b2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.528692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.529295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.529568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.529831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.530453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:58.530748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.530842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2b903-9637-4530-a917-ba518a42a553 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.532107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.548651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 339/672] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:58.548651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.575237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.575966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.576570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.576679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.587926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.590194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:17:58.606696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:58.607636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:17:58.608545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:17:58.608545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:17:58.608545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:17:58.608545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 334/673] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:58.608545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.616324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.656638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb3c0-0634-49e3-861b-f300c4cc4343 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.691279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.691884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.692161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.692434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.693199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=c4bd5bc35c41458b9ba20d87738cde16) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:58.693505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.694947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.701262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 340/674] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/role_assignments?user.id=c4bd5bc35c41458b9ba20d87738cde16&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:58.702043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.709621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.720385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f6c95-7775-4a12-ac8f-d5b27af0f598 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.764993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.765683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.765963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.766278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.766937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:58.767234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.769311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.792021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 335/675] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:58.793046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.849742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe613937-ffea-4b37-9e7c-8f2fb0ad6653 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.883080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:58.898998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.899639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.899941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.900213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.918795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.921456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:58.935057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d612fb-01d8-4f7c-bea8-c8fb7523165e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:58.941069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:58.941950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:58.942830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 341/676] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:58.943304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:58.987254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:58.987854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:58.988225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:58.988498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:58.989223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:58.989533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:58.990904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.003632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 336/677] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:59.004552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.016939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.074634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483ae9-7a1c-4d89-965a-beba21dc6b8b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.111033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.111682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.111958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.112223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.120185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.121658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.128602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030f6ea-bdad-4400-9182-c1e33c03e1a0 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:17:59.133422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:59.134328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:17:59.135266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:59.139179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 342/678] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:17:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:59.139985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.166835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.167689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.167969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.168239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.168984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:59.169290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.170944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.185326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 337/679] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:59.186251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.191970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.248235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c722adc-c5ed-443d-8356-7f1eed8b99fa None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.266765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.285536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.286163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.286449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.286686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.293886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.295423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:17:59.311240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:17:59.312333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:17:59.313489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:17:59.314306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 343/680] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:17:59.315253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.323584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3fe2c-7bba-4e19-81fa-910c0b3450ce None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.358648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.359250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.359523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.359793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.360522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:59.360853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.362904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91c443-54cd-46d7-87e4-f4423a0cfd38 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.379967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 338/681] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:59.381512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6397-635a-4b5a-8ca9-2a27cfaaaac5 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.381791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6397-635a-4b5a-8ca9-2a27cfaaaac5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.382062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6397-635a-4b5a-8ca9-2a27cfaaaac5 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.383442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 339/682] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:17:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:17:59.387942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba6397-635a-4b5a-8ca9-2a27cfaaaac5 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.434869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710c024-effc-43aa-a381-f274fe241baf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.442584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.472972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.473576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.473849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.474114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.474828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=06578427361b421fa275befc898a7f72) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:59.475126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.476736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.492793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 344/683] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/role_assignments?user.id=06578427361b421fa275befc898a7f72&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:59.492793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.512675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba6397-635a-4b5a-8ca9-2a27cfaaaac5 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.545041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.554412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.555295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.555671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.556042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.556857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:59.557236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.559013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e24462-82a5-4958-a8e5-4968bf69504a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.577305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 340/684] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:17:59.578941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.579400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.579743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.602304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:17:59.604017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:17:59.624715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890af89-b5b0-45e5-ba15-832c966cb95a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.667559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.668195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.668426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.668698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.692499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 341/685] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:17:59.693471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.723173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.757545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.759090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.824998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40572-93c7-4600-b6ad-4cb107db0d76 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:17:59.856480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:17:59.884792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:17:59.885386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:17:59.885665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:17:59.885937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:17:59.886674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=12a2363821f445db980712d3e491e791) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:17:59.886978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:17:59.889120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:17:59.897901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 345/686] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:17:59.905171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:17:59.908983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 342/687] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/role_assignments?user.id=12a2363821f445db980712d3e491e791&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:00.036467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:00.045569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.094120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dc569-19b5-4e1b-8399-027d9403a79d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.144542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.144542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.144832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.145833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.164658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:00.167161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:00.188620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:00.190312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:00.191213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:00.191213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:00.191213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:00.194016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 346/688] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:00.200607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.201280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c87f3-7d04-4179-9043-2ba27f638106 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.243033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.244052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.244534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.245064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.256924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:00.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:00.263632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:00.264497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:00.267407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 343/689] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:00.275181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.321751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6fcda-cf50-44bd-91a0-a2ab60b35eb5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.325976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:00.346811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:00.359731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.360341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.360618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.360927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.362452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:00.362995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:00.364494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:00.377482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 347/690] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:00.383191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.400926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a948dc-7ffd-4223-b765-ec971235ec59 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.444800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.445831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.446337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.446808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.447794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:00.448344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:00.449956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:00.466759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 344/691] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:00.485537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.500790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0997883-46e4-4cda-a8c0-f3b843a76e7b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.535985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.535985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.535985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.535985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.543250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:00.548817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:00.568582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:00.569795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:00.570881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:00.573532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 348/692] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:00.582057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.667749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91845cb-6333-49aa-9349-f7f330367108 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.669515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:00.675808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:00.742919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.743563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.743843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.744113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.766926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:00.768656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:00.786968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d5fd9-8bde-42c1-bc80-ff57286d6ed4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:00.793885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:00.795757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:00.797836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:00.797836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:00.797836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:00.800819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 345/693] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:00.802008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.832447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.833082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.833357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.833628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:00.834323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:00.834625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:00.836076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:00.865138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 349/694] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:00.873773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:00.959486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54eecd-5f63-4219-a298-485ad652f3e3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:00.996724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:00.998660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:00.999675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:00.999951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.000600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:01.000925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:01.003004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:01.033958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 346/695] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:01.042232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:01.062287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:01.083571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7377-ed17-4475-b20d-74830dfa6edb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:01.085168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:01.122104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:01.122892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.123147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.123417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.124138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=9d8eb54303a94ec38dbb9209c71381da) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:01.124469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:01.125929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d924987-e30f-4311-a061-586eeb8b62c1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:01.136894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 350/696] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:00 2022] GET /identity/v3/role_assignments?user.id=9d8eb54303a94ec38dbb9209c71381da&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:01.138246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2db5-2d91-4f43-be40-5eaafd77cd8e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.138517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2db5-2d91-4f43-be40-5eaafd77cd8e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.138792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2db5-2d91-4f43-be40-5eaafd77cd8e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.144892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 351/697] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:01.160728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.161092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.161298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.175534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acfbd8-a4ce-45ce-9aba-eeec2f933ed4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:01.201879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:01.203121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:01.218198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:01.219215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.220260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.220852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.230448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:01.232275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:01.246306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:01.247347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:01.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:01.251347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 347/698] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:18:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:01.257902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:01.347401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:01.379887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e42c3c-e5bd-4d8b-abe6-a8683b85487b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:01.402256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 352/699] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:01.418248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:01.419195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.419695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.420171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.421121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:01.421671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:01.423313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:01.443611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 348/700] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:18:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:01.449408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:01.554242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:01.560270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0c653-df8a-41e4-8d18-b86d2e9c9e0f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:01.599904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b0384-2e9e-42ca-b35d-ef49635c7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:01.600519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b0384-2e9e-42ca-b35d-ef49635c7896 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.600806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b0384-2e9e-42ca-b35d-ef49635c7896 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.601081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b0384-2e9e-42ca-b35d-ef49635c7896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.677260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b0384-2e9e-42ca-b35d-ef49635c7896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:01.678722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b0384-2e9e-42ca-b35d-ef49635c7896 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:01.732135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:01.783767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:01.792591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 353/701] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:18:01.793926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501a6fc-4499-4a89-bb3c-b2d87c6256e9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.794212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501a6fc-4499-4a89-bb3c-b2d87c6256e9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.794486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501a6fc-4499-4a89-bb3c-b2d87c6256e9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.795854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 354/702] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:01.801500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501a6fc-4499-4a89-bb3c-b2d87c6256e9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:01.874274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b11e55-2577-4c70-9645-b583f88ebe61 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:01.926511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:01.933555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:01.935184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.936013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.936793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.948940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:01.949961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501a6fc-4499-4a89-bb3c-b2d87c6256e9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:01.950492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-8929892c-fd81-468e-ba68-3920bd523f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:01.966863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:01.967845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:01.968869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:01.968869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:01.968869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:01.973259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 349/703] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:01.974909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.975106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.975337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:01.996203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:01.997569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:01.997828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be115068-6844-4416-b545-84ec083f941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:01.998685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be115068-6844-4416-b545-84ec083f941a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:01.999158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be115068-6844-4416-b545-84ec083f941a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:01.999930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be115068-6844-4416-b545-84ec083f941a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.016029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be115068-6844-4416-b545-84ec083f941a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.018450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be115068-6844-4416-b545-84ec083f941a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-be115068-6844-4416-b545-84ec083f941a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:02.034997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:02.035986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:02.036894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:02.036894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:02.036894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:02.040743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 355/704] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:02.041582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be115068-6844-4416-b545-84ec083f941a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.094728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:02.097449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 350/705] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:02.099121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.200360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be115068-6844-4416-b545-84ec083f941a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.225388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f016-3e17-49c7-86f3-178c1845f028 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.243042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:02.270017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.271086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.271567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.272201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.273197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:02.273789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.275475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.277882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.280060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.282424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.282907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.284314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:02.287613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.289339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.302475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 351/706] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:02.307872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.315149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 356/707] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:02.321204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.422726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a429-80ef-490f-8842-2cabad41ec6f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.468876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.469971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.470260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.470540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.478377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.479831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-5942c984-72e1-460d-93e6-1b55edb00691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:02.491069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:02.491963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:02.492900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 352/708] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:02.493433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d979-19e5-4112-acc8-b9320bfe762e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.493828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d979-19e5-4112-acc8-b9320bfe762e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.493929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d979-19e5-4112-acc8-b9320bfe762e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.496808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 353/709] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:02.496808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d979-19e5-4112-acc8-b9320bfe762e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.502648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8c588-4925-4c00-a322-a4065216a7b3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.513524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:02.537173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:02.541232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.541906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.542183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.542452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.543095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.544528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.566852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 357/710] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:18:02 2022] POST /identity/v3/users => generated 254 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:18:02.567679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.653511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d979-19e5-4112-acc8-b9320bfe762e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.701865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4576edd-097b-4af3-823b-d92e2b7a86f3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.711007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.712027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.715701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.715701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.715701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.715982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-f000479b-826b-4f93-b296-775cab712fe7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:02.728332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:02.729467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:02.730363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 354/711] 10.222.0.81 () {56 vars in 1129 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:02.731168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.731168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.731295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.750306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.750917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.751199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.751466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.752715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:02.752949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:02.753091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.754031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:02.754581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.770312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 358/712] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:02.771027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.810020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:02.842032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 355/713] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:02.842938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:02.909558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:02.923197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424a9b6-22d2-4c20-b54f-5ca335260342 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:02.966207 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:02.966820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:02.967147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:02.967418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:02.968043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:02.968340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:02.970150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:02.976106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 359/714] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:02.981984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:03.088200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022a2af-87d4-4359-a547-9ed1cf1990e0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:03.156224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:03.157028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.157170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.157433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.158233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=b8afb31c267946b7851cb1c22b62b6cb) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:03.158586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:03.160020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:03.175630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 356/715] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/role_assignments?user.id=b8afb31c267946b7851cb1c22b62b6cb&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:03.240874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:03.339819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:03.354876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:03.366740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc3d97-3e36-400f-84e5-aa9b310e5ec7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:03.369032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d42b4-099f-4b8e-b0d6-7b6b87c64022 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:03.404236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:03.404901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.405180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.405452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] PATH_INFO: `/v3/services/network` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.406042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:03.407519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:03.417462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:03.418539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:03.419798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 357/716] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:03 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:03.425304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:03.436780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:03.442908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.443199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.443478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.444127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:03.444424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:03.445931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d71f4-93be-49b5-9861-873a56784a39 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:03.453772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 360/717] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Aug 24 03:18:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:03.599649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f31d1-16e6-45ef-afbe-ca9763ba95aa None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:03.647236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:03.648194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.648675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.649172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.650044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] RBAC: Adding query filter params (name=network) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:03.650611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:03.652249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:03.657503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 358/718] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:03.661648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d809e-7742-4e04-af81-78a6ac3d6a6b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.661648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d809e-7742-4e04-af81-78a6ac3d6a6b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.661873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d809e-7742-4e04-af81-78a6ac3d6a6b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.663217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:03.663437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 361/719] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:03.669873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.670152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.670428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.688970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:03.694280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:03.696359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:03.782039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 362/720] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:18:03.823608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67617503-39a5-4c31-86c2-46e118101a65 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:03.871868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:03.872476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:03.872750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:03.873050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:03.873687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] RBAC: Adding query filter params (type=network) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:03.873987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:03.875494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:03.887691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 359/721] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:04.120290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:04.209686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:04.232728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060eaf6f-2ae8-424c-ba51-1dd565f710b0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:04.272203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:04.272857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:04.273133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:04.273405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:04.289777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:04.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:04.295457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:04.296865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 363/722] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:04.301988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:04.416202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:04.437595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e9903-951a-467f-a997-0b79d726a3de None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:04.485962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:04.486735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:04.487015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:04.487290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:04.487960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:04.488263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:04.489661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:04.498480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 360/723] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:04.506366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:04.611040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5388e14-fa1b-49e9-b792-5d57aa793d15 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:04.654730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:04.655366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:04.655646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:04.655917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:04.669914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:04.671389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:04.673185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:04.697497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:04.698454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:04.699401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:04.699401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:04.699401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:04.699401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 364/724] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:04.704432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:04.860313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd75066-14e1-45be-b10c-819550b95827 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:04.916792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:04.916792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:04.916792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:04.916792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:04.917160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:04.917160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:04.917160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686366-0782-4329-8c1b-3539e0225f66 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:04.938028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 361/725] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:04.943828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:05.034393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a6643-5488-4d3e-b743-9377d351c07d None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.034678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a6643-5488-4d3e-b743-9377d351c07d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.035615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a6643-5488-4d3e-b743-9377d351c07d None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.037970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 362/726] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:05.043301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.043652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.043998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.076822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:05.082922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:05.085742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:05.123184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb30c-3553-40eb-a509-5c00c349c5e3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:05.178904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:05.179813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.180237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.180656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.194917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:05.196602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:05.217350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:05.218555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:18:05.219747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:05.224474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 363/727] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:05.225286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 365/728] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:05.232415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:05.372115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd4f0f-dbff-42b6-a5a6-2d6728f855c6 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:05.420003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:05.420534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.420834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.421112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.421852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:05.422153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:05.423818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:05.447189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 364/729] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Aug 24 03:18:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:05.449143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:05.453624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:05.535931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:05.553288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:05.616364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d947d-8acd-4d5a-a350-7bd07dbb0b0d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:05.689050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c3ddc-ad98-4a9e-b7fb-6c7c287d7735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:05.690183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c3ddc-ad98-4a9e-b7fb-6c7c287d7735 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.690665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c3ddc-ad98-4a9e-b7fb-6c7c287d7735 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.691134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c3ddc-ad98-4a9e-b7fb-6c7c287d7735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.748080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddd5f6-314d-4848-9554-d64ce2be51b4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:05.794621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745e7f6-b21e-4e4b-8445-0c1495638970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:05.795679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745e7f6-b21e-4e4b-8445-0c1495638970 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.796171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745e7f6-b21e-4e4b-8445-0c1495638970 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.796557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c3ddc-ad98-4a9e-b7fb-6c7c287d7735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:05.796934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745e7f6-b21e-4e4b-8445-0c1495638970 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.798345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c3ddc-ad98-4a9e-b7fb-6c7c287d7735 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:05.808706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745e7f6-b21e-4e4b-8445-0c1495638970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-5745e7f6-b21e-4e4b-8445-0c1495638970 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:05.814017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:05.814958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:18:05.815820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:05.818517 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 365/730] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:18:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:05.820020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b691d-66b7-4e8c-8532-75fe3d738042 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.820504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b691d-66b7-4e8c-8532-75fe3d738042 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.821007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b691d-66b7-4e8c-8532-75fe3d738042 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.823054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 366/731] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:05.823961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b691d-66b7-4e8c-8532-75fe3d738042 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:05.875295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:05.897854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:05.944899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 366/732] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:18:05.944899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8784d27-ff8b-4cf5-9b6d-05cca3b8a983 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:05.944899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8784d27-ff8b-4cf5-9b6d-05cca3b8a983 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:05.944899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8784d27-ff8b-4cf5-9b6d-05cca3b8a983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:05.964693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b691d-66b7-4e8c-8532-75fe3d738042 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:05.966107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a8784d27-ff8b-4cf5-9b6d-05cca3b8a983 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:05.967285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a8784d27-ff8b-4cf5-9b6d-05cca3b8a983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:06.004524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.005163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.005441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.005712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.006381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:06.006682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.008066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.016160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 367/733] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:18:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:06.016981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.053062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 367/734] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:06.054261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cd37a-a591-444c-989d-c5e860a2ed05 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.054547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cd37a-a591-444c-989d-c5e860a2ed05 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.054819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cd37a-a591-444c-989d-c5e860a2ed05 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.056116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 368/735] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:06.056833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cd37a-a591-444c-989d-c5e860a2ed05 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.136751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194d1ea-9302-489d-b26c-cafa78f8e9a4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.165222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.165809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.174792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.175429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.175713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.176124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.188557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cd37a-a591-444c-989d-c5e860a2ed05 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.188723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.190949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-5635cab9-006d-4d45-b06d-87d5f7d40bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:06.212352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:06.213541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:06.214550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:06.214550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:06.214550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:06.214550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 368/736] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:06.214550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.214832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.214979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.237756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:06.238983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:06.244071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.244923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.245343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.245776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.261582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.263826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:06.283973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:06.285124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:06.285998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:06.285998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:06.285998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:06.285998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 369/737] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:06.290752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.332106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 369/738] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:06.333053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.396864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dba21-767d-40bd-ad78-58b7f4cbc045 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.432554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.433188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.433468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.433738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.434358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.434786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:06.434786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.436098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1893da-af22-4a36-9931-0f3c82ea0ebd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.438515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.438929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd3a45-d081-4054-a719-ca69191ba1a1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.450259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 370/739] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:06.451453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdee3c-916f-429b-aec7-23b0d3a0c3dc None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.451732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdee3c-916f-429b-aec7-23b0d3a0c3dc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.452005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdee3c-916f-429b-aec7-23b0d3a0c3dc None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.453377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 371/740] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:06.453912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdee3c-916f-429b-aec7-23b0d3a0c3dc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.483769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.484288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.484568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.484875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.485521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:06.485822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.487723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.504084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 370/741] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:06.504084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.568482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdee3c-916f-429b-aec7-23b0d3a0c3dc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.603978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c388ae-6e61-4911-b953-9e0f4c806a8b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.605277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240a5d-eb3b-476c-b8ac-0d78eb5551c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.605903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240a5d-eb3b-476c-b8ac-0d78eb5551c9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.606187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240a5d-eb3b-476c-b8ac-0d78eb5551c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.606456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240a5d-eb3b-476c-b8ac-0d78eb5551c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.618236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240a5d-eb3b-476c-b8ac-0d78eb5551c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7f240a5d-eb3b-476c-b8ac-0d78eb5551c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:06.622928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:06.623862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:06.624704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 372/742] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:06.625689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.625689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.625946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.639859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.640504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.640790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.641096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.644094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:06.645529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:06.648077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.650238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:06.661170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:06.662222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 371/743] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:06.663504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.683772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.700839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.717804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 373/744] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:18:06.719188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.768072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3fa9-be82-4167-819c-f5b435421c5b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.804676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.805332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.805602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.805897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.813586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.815232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:06.826696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:06.827661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 372/745] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:06.828620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.850655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646518a-2d51-4c63-a783-fae436c56801 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:06.889374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:06.889979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:06.890258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:06.890592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:06.891242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:06.892630 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:06.916343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 374/746] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:06 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:06.917558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:06.925805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:06.946069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6742e8-4109-463e-8b2d-c36374e311f2 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.000783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.003391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.004022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.004302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.004574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.005289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:07.005591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.007020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.025318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 373/747] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:07.025318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.076559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e590d43-93b1-4456-80f4-03bba31d0582 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.116674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.117316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.117592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.117860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.118551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:07.118853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.120286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.135063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 375/748] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:07.135879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.148572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281f630-1e2b-405d-8974-e7a8a1075e03 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.186829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.187449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.187715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.187990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.188728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:07.189063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.190694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.194540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.210883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 374/749] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:07.211710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.251061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8aa0c-b06e-4a96-940c-aea2ceacae46 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.260338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.301855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.303050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.303466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.303881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.304900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.307285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.332823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 376/750] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:07 2022] POST /identity/v3/services => generated 240 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:07.332823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.352748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c91e96-5b95-49dc-8c5f-d7f19af3f49c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.402071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.402747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.403025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.403293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.417190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.418781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:07.436443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:07.437887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:07.438769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:07.438769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:07.438769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:07.438769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 375/751] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:07.438769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.473233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7796ec-4641-4a26-a233-35e81b938a5d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.491971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.527896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.528496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.528915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.529956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.529956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=9d8eb54303a94ec38dbb9209c71381da) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:07.531771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.534237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.542491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 377/752] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/role_assignments?user.id=9d8eb54303a94ec38dbb9209c71381da&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:07.543284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.572225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2d973-8883-4659-902a-d1bcbd7677c4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.585482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.626188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.627662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.628645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.629019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.690927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376dc9-b56a-493b-b678-9629982f2c00 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:07.736799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.738337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.740127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:07.741179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:07.742622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:07.743102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:07.743941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:07.744472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:07.746060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:07.760079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 378/753] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:07.768820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.824725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.918802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 376/754] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:18:07.924950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:07.938607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:07.964236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d421c5-a650-4392-9ebd-508fc85e7eeb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.042669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.042669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.042669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.042669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.052119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae370206-48ce-4349-b939-cc9f37fde4b6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.062190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.063899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:08.086284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:08.087273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:08.088220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 379/755] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:08.094491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:08.097964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.098880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.099357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.099828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.120795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.120795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.137112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:08.147614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:08.148593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:08.149607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:08.149607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:08.149607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:08.153019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 377/756] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:08.161807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:08.294520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f5773-b264-4e20-b267-208c4691c8fb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.331127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a294-b8fe-46ec-b195-24fde7424420 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.370724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.371738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.371981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.372460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.372581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.372863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.373129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.373495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.373850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:08.374146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.374734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:08.375225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.375703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.377559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.397390 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 378/757] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:08.406203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:08.411270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 380/758] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:08.416234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:08.432278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:08.535933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:08.540215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d4dc-8264-470e-9f60-ebb6b20d9a1a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.581956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.583416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.583941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.584844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.597946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.600077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:08.618263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:08.619275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:08.620166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:08.620166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:08.620166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:08.622946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 381/759] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:08.627742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:08.636383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e65663-bdb0-4ce7-b63a-c280e5395647 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.741287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:08.744237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d81b2-7b6b-45a5-bab6-3e997a20ab9a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:08.754722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.755680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.756172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.756670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.769306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.771105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:08.788605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:08.789587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:08.790525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:08.793168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 379/760] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:08.800126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:08.817484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:08.818502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:08.818998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:08.819526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:08.820365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:08.820909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:08.822695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:08.846679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 382/761] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:08.854204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:09.008365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9df5aa-708f-407a-b978-3d958aa6ba6b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.013977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5530ca-a46e-42c7-bb44-facd8181b099 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.023180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:09.046957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.047555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:09.048000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.048000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.049144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.050266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:09.050583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.052238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a1c47-74b7-448e-8048-ca5152020f5d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:09.061677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.064019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.064626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.065006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.067147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 380/762] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:09.068463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58bf66-f936-49d8-83c3-12120ad64f0f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.068840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58bf66-f936-49d8-83c3-12120ad64f0f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.069178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58bf66-f936-49d8-83c3-12120ad64f0f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.070467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 381/763] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:09.075029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.075485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58bf66-f936-49d8-83c3-12120ad64f0f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:09.076652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bddc1659-27dd-4b8c-a3dc-7aaa4864fd57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:09.091548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:09.092512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:09.093467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:09.093905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 383/764] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:09.095062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.096009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.096285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.119269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:09.121304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:09.192673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58bf66-f936-49d8-83c3-12120ad64f0f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.195821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 384/765] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:09.196689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:09.255374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.256065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.256407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.256744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.257629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=cbd35279778f4858808385011249d9f5) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:09.257997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.259480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:09.272879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 382/766] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:09 2022] GET /identity/v3/role_assignments?user.id=cbd35279778f4858808385011249d9f5&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:09.308682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af573ddf-6534-4979-a489-3f9e747352d5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.321616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:09.349279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.349891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.350233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.350508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.351255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:09.351562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.353311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:09.367719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 385/767] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:09.373403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:09.447599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:09.522122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02668ca3-7329-4401-b842-6c2073c498db None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.567356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58791-c0d4-4586-a1bb-95c20396e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.568820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58791-c0d4-4586-a1bb-95c20396e864 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.569304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58791-c0d4-4586-a1bb-95c20396e864 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.569778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58791-c0d4-4586-a1bb-95c20396e864 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/b8afb31c267946b7851cb1c22b62b6cb/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.578101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58791-c0d4-4586-a1bb-95c20396e864 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=b8afb31c267946b7851cb1c22b62b6cb, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.580314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58791-c0d4-4586-a1bb-95c20396e864 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:09.594259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:09.607792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 383/768] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:18:09 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/b8afb31c267946b7851cb1c22b62b6cb/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:18:09.737684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00236ca7-dadb-41a8-bb0d-2e4085d05bee None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.789705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.790348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.790623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.790892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.791538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.792944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:09.811008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:09.812068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:09.812932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 386/769] 10.222.0.81 () {56 vars in 1129 bytes} [Wed Aug 24 03:18:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:09.818740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:09.819224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809d347-7a60-4f64-b596-ad9bb9231b70 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.819529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809d347-7a60-4f64-b596-ad9bb9231b70 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.820010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809d347-7a60-4f64-b596-ad9bb9231b70 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.821664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 384/770] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:09.827302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6044-ba18-498c-ba5b-6e4b584b3203 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.827611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6044-ba18-498c-ba5b-6e4b584b3203 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.827858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6044-ba18-498c-ba5b-6e4b584b3203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.869829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-717a6044-ba18-498c-ba5b-6e4b584b3203 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:09.871049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-717a6044-ba18-498c-ba5b-6e4b584b3203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:09.903256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:09.957865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bc82b-2310-473c-a8a3-b2fe95927bfe None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:09.963586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 385/771] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:09.964996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398358a-93ff-4e5d-ad53-0ee9defbfeae None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.965281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398358a-93ff-4e5d-ad53-0ee9defbfeae None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.965554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398358a-93ff-4e5d-ad53-0ee9defbfeae None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.967176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 386/772] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:09.978014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.978293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.978568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.996289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:09.996883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:09.997161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:09.997259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:09.997478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:09.998151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:09.998364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:09.998460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:09.999802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.006469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 387/773] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Aug 24 03:18:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:10.011895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.035186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:10.081324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 387/774] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:10.144906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:10.149340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7532c9-d46b-47d2-846d-6c1a50277046 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.186135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.186732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.187001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.187266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.187885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:10.188175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:10.190351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.195597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 388/775] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:10.200989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.343530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4326f50-aef3-4b7c-9d20-45be0899284f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.352747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.406397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.407018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.407302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.407570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.408348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] RBAC: Adding query filter params (interface=public, service_id=2ee1b3bedb2548a2b2015a88945bbe96, region_id=RegionOne) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:10.408654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:10.410221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.424924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 388/776] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/endpoints?service_id=2ee1b3bedb2548a2b2015a88945bbe96&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:10.434060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.466771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbe17f-df04-44de-8c4c-a197c9141574 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.493041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:10.515747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.516751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.517925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.518429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.533890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:10.540948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:10.542117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:10.543167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:10.546390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 389/777] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:10.547349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.574491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:10.605372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c2817-8ee4-4a40-b583-eec321f8b986 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.643053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.643996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.644574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.645134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.645931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:10.647479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.653273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 389/778] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:10.654268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.669988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3223330-2629-4f0b-aeb5-87cdfa8a43ca None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.715396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.716400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.716950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.717417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] PATH_INFO: `/v3/services/network` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.718224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:10.719956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:10.731881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:10.733043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:10.733859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:10.736899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 390/779] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:10.754127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.857210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:10.867942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3389d4-83ba-4e7c-b5e2-c7618346091a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.915537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.916145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.916423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.916699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.917413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:10.917716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:10.919103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.922019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:10.929645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 390/780] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:10.938568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:10.942090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80759e1a-624f-4252-8d9b-a1b51ad15abe None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:10.979294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:10.979899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:10.980220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:10.980489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:10.982139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] RBAC: Adding query filter params (name=network) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:10.982139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:10.982932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:10.991939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 391/781] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:11.004792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:11.136302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed2f00-dfc1-4301-b0d2-73e762691609 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.138989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd8ae-165c-432f-9c09-817296dd258a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.176157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.176875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.177332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.177664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.178609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] RBAC: Adding query filter params (type=network) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:11.178992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.181043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.186539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 392/782] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:11.197039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:11.214260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.215026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.215304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.215576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.232949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:11.236882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.243606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.258736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:11.277784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:11.278811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:11.279696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:11.279696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:11.279696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:11.280796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 391/783] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:11.287905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:11.391748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d2f6-16ae-4e01-8767-18caf32cce78 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.407559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afd10e-11c3-474d-b4de-162cb0425d83 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.447018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.448004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.448474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.448983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.450051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:11.450582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.452298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.467455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 392/784] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:11.473955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:11.493496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.494171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.494394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.494665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.495437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] RBAC: Adding query filter params (interface=public, service_id=6cafcc1c1918460280b7edfc998517b4, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:11.495747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.497158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.515588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 393/785] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:11 2022] GET /identity/v3/endpoints?service_id=6cafcc1c1918460280b7edfc998517b4&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:11.522773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:11.545204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:11.590479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c82c8-3f63-4cc7-9978-bf491a0c4040 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.637519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.639469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.640230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.640747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.651731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.654423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.663110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-e0722bcf-2571-46f2-bd52-902d587a73e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:11.672585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:11.674164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:18:11.675111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:11.678149 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 393/786] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:11.683074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68aaec-ddc4-442f-a5fc-615cd2e9fff5 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.683074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68aaec-ddc4-442f-a5fc-615cd2e9fff5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.683074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68aaec-ddc4-442f-a5fc-615cd2e9fff5 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.684906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 394/787] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:11.690098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68aaec-ddc4-442f-a5fc-615cd2e9fff5 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:11.699063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f694-5f5b-40e7-a2a0-f528d5587dec None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.742862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92483f0b-65a1-4499-ac2a-7ec4fe16a763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.743959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92483f0b-65a1-4499-ac2a-7ec4fe16a763 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.744240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92483f0b-65a1-4499-ac2a-7ec4fe16a763 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.744512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92483f0b-65a1-4499-ac2a-7ec4fe16a763 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.745159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92483f0b-65a1-4499-ac2a-7ec4fe16a763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.746950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92483f0b-65a1-4499-ac2a-7ec4fe16a763 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.760799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 394/788] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:11 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:11.760799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.760799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.760799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.777190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:11.778873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:11.897020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68aaec-ddc4-442f-a5fc-615cd2e9fff5 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:11.926496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 395/789] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:11.938053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:11.954938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:11.955653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:11.955955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:11.956246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:11.957058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:11.957467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:11.958955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4939c-b1fb-436d-93bd-c603dd0604e5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:11.978620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 395/790] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Aug 24 03:18:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:11.984017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.032052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32a7df-3fe9-4c3b-87d5-07e31236b875 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.033280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32a7df-3fe9-4c3b-87d5-07e31236b875 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.033754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32a7df-3fe9-4c3b-87d5-07e31236b875 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.035262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 396/791] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:12.040949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.041228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.041501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.075729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:12.076960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:12.091404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:12.136790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c47667-db38-44eb-bd83-fd565c9455ac None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:12.204557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:12.205413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.205413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.205726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.208723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 397/792] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:12.232580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:12.234376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:12.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:12.255328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:12.256193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:12.256193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:12.256193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:12.257977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 396/793] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:12.258342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.267404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.431755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f97b22-e98c-4cf9-b0f7-aaeb8d07759d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:12.463943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408aaf-511a-4fe8-ab1b-ccfabdd54630 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:12.525170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:12.547135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:12.547764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.548042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.548920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.549958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:12.550348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:12.551751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:12.573699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:12.573891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 397/794] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:12.574723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.615990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:12.616802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.617121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.617419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.630499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:12.635389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:12.636285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:12.637150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 398/795] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:12.638251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.708526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e28dca-3383-489e-a8d8-0279b8977f41 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:12.746025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:12.746625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.746997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.747361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.758563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:12.763274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:12.764353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:12.765406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:12.770418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6c43b-83e6-4624-adaa-30b60d47537c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:12.772417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 398/796] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:12.773817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.810455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:12.811411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.811892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.812374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.820608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:12.822346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:12.823761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:12.837822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:12.838735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:12.839607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:12.842092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 399/797] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:12.843062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.874144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:12.886998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e37f-6f53-43fe-a829-e4e44bd880f1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:12.926461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:12.927201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:12.927383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:12.927644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:12.928353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:12.928648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:12.930096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:12.938695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 399/798] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:12.939898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:12.964297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ecc2a-0c3e-49c0-8b13-a287a45bef20 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.004341 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.005491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.005976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.006448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.007344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:13.007845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.009656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.019411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 400/799] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:13.020555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.052876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5674d40-8845-461e-9b8f-0ad47d511021 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.060174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.090045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.090660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.090931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.091194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.091889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:13.092194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.093697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.108892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 400/800] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:13.108892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.129710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.142522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b22c1-f82a-452e-bc2f-5e60751fe6bc None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.179983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.180596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.180894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.181162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.192354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.193946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-65660dbd-09a9-4936-a86d-8a69160087cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:13.209208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:13.210165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:13.211883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:13.211883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:13.211883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:13.211883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 401/801] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:13.211883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390d0d3-5926-4ff2-9c18-a7e484562bd7 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.212164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390d0d3-5926-4ff2-9c18-a7e484562bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.212231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390d0d3-5926-4ff2-9c18-a7e484562bd7 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.217004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 402/802] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:13.217004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390d0d3-5926-4ff2-9c18-a7e484562bd7 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.222865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4c48-ea4b-4d15-bd41-bd7986a7264c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.258069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.258619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.258898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.259169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.271287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.272747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:13.287810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:13.289098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:13.290446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:13.290446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:13.290446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:13.290446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 401/803] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:13.290446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.311095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.334037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390d0d3-5926-4ff2-9c18-a7e484562bd7 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.371721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe9327-5b2d-4da1-a117-c8a9f1f3de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.372371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe9327-5b2d-4da1-a117-c8a9f1f3de9c None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.372648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe9327-5b2d-4da1-a117-c8a9f1f3de9c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.372952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe9327-5b2d-4da1-a117-c8a9f1f3de9c None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/9d8eb54303a94ec38dbb9209c71381da/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.374887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe9327-5b2d-4da1-a117-c8a9f1f3de9c None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=9d8eb54303a94ec38dbb9209c71381da, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.376323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe9327-5b2d-4da1-a117-c8a9f1f3de9c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.400793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 403/804] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:18:13 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/9d8eb54303a94ec38dbb9209c71381da/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:18:13.400793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.400793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.400793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.405225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.419681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:13.420936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:13.425602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1934a1-b0b2-4b68-a6ca-a1cc6241481f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.464034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.464851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.465230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.465601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.466359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:13.466759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.468378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.487221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 402/805] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:13.487221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.518676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 404/806] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:13.520058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.604330 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.630566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc3fa-ff6f-47b2-b23d-3c85503e251b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.671478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.672378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.672888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.673361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.674127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:13.675198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.676654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.681210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f0fdf-8bd0-4a8e-ab32-452dcb8cae75 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.691240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 403/807] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:13.699687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.765763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.780233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.780826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.781110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.781382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.801939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.803414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.812863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9899b58-e9e6-4f5f-ae84-d08084ffed14 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:13.824219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:13.825294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:13.826488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 405/808] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:13.842112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.861444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.862386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.862810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.863271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.871773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.873499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:13.875577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:13.887749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:13.888982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:13.890231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:13.892651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 404/809] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:13.898301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:13.952830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246d751-85cf-4ff0-82fb-1d65d685696b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:13.990470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:13.991104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:13.991378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:13.991645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:13.992225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:13.993622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:14.002784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:14.003793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:18:14.004618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.008706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 406/810] 10.222.0.81 () {56 vars in 1129 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:14.009580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.012911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec232eb4-f739-4f6c-8eea-853729637fd4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.051823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.052343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.052638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.052911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.053640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:14.053942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.055369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4449ab-7468-4284-9490-ea1d69b8bba1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.073155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 405/811] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Aug 24 03:18:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:14.074666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523b979-d48a-4c06-a55a-e76e56633741 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.075184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523b979-d48a-4c06-a55a-e76e56633741 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.075655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523b979-d48a-4c06-a55a-e76e56633741 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.077083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 406/812] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:14.077952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523b979-d48a-4c06-a55a-e76e56633741 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.078420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.130110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96b5c-c2dc-4822-aafa-a76ef552de19 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.141286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.165708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.166298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.166572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.166836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.167562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:14.167861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.169301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.187759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 407/813] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:14.188574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.200872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523b979-d48a-4c06-a55a-e76e56633741 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.246045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.246751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.247031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.247305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.247937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:14.248350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.249877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487671fd-25b6-44cd-9c9a-80ac906e246d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.260077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 407/814] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:14.261230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.261511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.261782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.279133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:14.280329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:14.295601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20ba5-a027-4eb5-a1c7-002ff62b488b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.312938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.337313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.337926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.338196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.338461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.349292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.350715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.364894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 408/815] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:14.365268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:14.366162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:14.367060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:14.367060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:14.367060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.367060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 408/816] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:14.367060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.368525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.425919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.481973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469689c5-c15b-4ba3-a1ba-fb26b6618d73 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.496216 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc321d-eef9-43a4-a478-4f1345c7411f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.520099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.520808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.521255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.521598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.535580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.536188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.536465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.536735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.537393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:14.537693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.539267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.549918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 409/817] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:14.550926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.585975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.598934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.600856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.654995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.665609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffa25d-ef19-416c-8891-cf39453bb872 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.703533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.704202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.705076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.705492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.706251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:14.706550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.707909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.717486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 409/818] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:18:14.717486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.724242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 410/819] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:14.725258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.819883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:14.824477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c65efa-052f-42a8-9d3a-6d328fcf1cde None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.831754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a078b-6536-483c-b217-e1a94ef27389 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:14.865516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.866138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.866417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.866689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.867343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.867523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:14.868108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:14.868382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:14.868650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] PATH_INFO: `/v3/services/network` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:14.868749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.869263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:14.870602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:14.879989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:14.881081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:14.881899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 411/820] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:14.882694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.896668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 410/821] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:14.897889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:14.908464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.027721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f440929-0929-47bc-b3e0-3ff84eeececf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.057834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc89e64-7e93-48c1-93c9-61b94c21b6eb None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.076222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.077368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.077851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.078489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.087406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.089291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:15.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:15.103351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:15.104225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 411/822] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:15.104703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.108023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.110684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.110684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.110684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.127253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.129348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:15.146138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:15.147123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:15.153262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:15.153262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:15.153262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:15.153262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 412/823] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:15.153262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.160662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.186426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.233158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a7f75-2215-4570-b08a-46d73612aac2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.269000 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.269778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.270152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.270516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.271241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:15.271634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.273307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.278083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 412/824] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:15.281169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32ed4a-2856-4088-8ba3-812e36403067 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.284053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.325876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.326474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.326744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.327005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.327688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:15.327981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.329666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.348212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 413/825] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:15.348802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.419254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.435767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654d9c7-6653-4da0-a6bf-9e17287300b8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.462089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74695ab-63e2-49a7-a60a-b280fc2b461f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.472283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.490543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.491424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.491702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.491974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.492790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:15.493489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.495308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.501301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.502086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.502403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.502667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.503299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] RBAC: Adding query filter params (type=network) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:15.503598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.505505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.514748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 413/826] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:15.515699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.516090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 414/827] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:15.517531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.624605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53805e9e-cbd8-496e-bc5c-89a00635d12b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.624997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d8a0c-eb6c-4f46-be67-71e1e2a2ecd0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.654221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.661759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635384e-69f8-41f8-b5af-6b20d943ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.661940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.662389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.662490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635384e-69f8-41f8-b5af-6b20d943ec10 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.662666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.662761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635384e-69f8-41f8-b5af-6b20d943ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.662939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635384e-69f8-41f8-b5af-6b20d943ec10 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/cbd35279778f4858808385011249d9f5/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.663042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.669906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.671292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.678884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635384e-69f8-41f8-b5af-6b20d943ec10 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=cbd35279778f4858808385011249d9f5, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.680345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635384e-69f8-41f8-b5af-6b20d943ec10 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:15.684427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:15.685690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 415/828] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:15.687057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.705687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 414/829] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:18:15 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/cbd35279778f4858808385011249d9f5/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:18:15.705687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d8b46-7841-4304-bdca-26171b9b5670 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.705687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d8b46-7841-4304-bdca-26171b9b5670 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.705687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d8b46-7841-4304-bdca-26171b9b5670 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.705687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 415/830] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:15.705687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d8b46-7841-4304-bdca-26171b9b5670 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.748650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:15.824494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f2b9b-fd8b-4ca1-a0ba-c3e62a9818bf None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.832886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d8b46-7841-4304-bdca-26171b9b5670 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:15.872983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.873681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.873957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.874231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.875005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.875107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:15.875704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.876018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.876263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.876910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.877524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:15.877852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:15.879294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba4395-9d8c-4da0-8c4e-64b500be1c96 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:15.894900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 416/831] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:15.896570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3900417-3477-4273-856b-7675813c1ccc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:15.897092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3900417-3477-4273-856b-7675813c1ccc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:15.897746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3900417-3477-4273-856b-7675813c1ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:15.909578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 416/832] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:15 2022] POST /identity/v3/endpoints => generated 337 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:15.910301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:15.922493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c3900417-3477-4273-856b-7675813c1ccc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:15.924257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c3900417-3477-4273-856b-7675813c1ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:15.982373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:16.016510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 417/833] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:16.072677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248be10-7cf0-4133-9686-c1eac895df29 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:16.129552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673631c-907e-4040-8362-6416acd631fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:16.130528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673631c-907e-4040-8362-6416acd631fd None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:16.131006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673631c-907e-4040-8362-6416acd631fd None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:16.131516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673631c-907e-4040-8362-6416acd631fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:16.132442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673631c-907e-4040-8362-6416acd631fd None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=b8afb31c267946b7851cb1c22b62b6cb) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:16.132995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673631c-907e-4040-8362-6416acd631fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:16.134621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673631c-907e-4040-8362-6416acd631fd None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:16.147968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 417/834] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:15 2022] GET /identity/v3/role_assignments?user.id=b8afb31c267946b7851cb1c22b62b6cb&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:16.327512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3900417-3477-4273-856b-7675813c1ccc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:16.356434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:16.443366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3900417-3477-4273-856b-7675813c1ccc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:16.480504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:16.481600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:16.482068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:16.482526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:16.493780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:16.499379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:16.501121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:16.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:16.506237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 418/835] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:16.512805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673631c-907e-4040-8362-6416acd631fd None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:16.630292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673631c-907e-4040-8362-6416acd631fd None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:16.641079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:16.666587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:16.667385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:16.667952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:16.668223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:16.668938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:16.669239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:16.670609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:16.680638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 418/836] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:16.685838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:16.777345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:16.797261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def088-73f5-4358-80bf-d40d1992c9f2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:16.841629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:16.842613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:16.843118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:16.843610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:16.854759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:16.856453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-767a5ced-890d-4b68-8855-0162bd014957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:16.871706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:16.872659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:16.873570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:16.873570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:16.873570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:16.875672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 419/837] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:16.882134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.005003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801ddf4-9f8c-4365-a44a-ab7781c26738 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.041970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.042596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.042873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.043143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.043802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:17.044103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.045540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.047682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fae11-161d-4327-9e10-9c597df6142d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.047975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fae11-161d-4327-9e10-9c597df6142d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.048808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fae11-161d-4327-9e10-9c597df6142d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.050641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 420/838] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:17.055863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.056144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.056419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.059563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 419/839] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:17.065807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.079014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:17.080712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:17.101961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:17.166492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 421/840] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:17.190511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374beec4-f9ce-4391-85ee-1de1dc22ef04 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.228563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.229541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.230055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.230497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.238856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.240501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:17.252850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:17.253773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:18:17.254747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:17.256952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 420/841] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:17.262774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.332159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:17.374119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cae004-3e5c-4bb4-8484-d8d21b5484fd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.412210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.413147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.413565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.414002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.414874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:17.415353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.417046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.436072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 422/842] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:17.443541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.457325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.543867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:17.554709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96d8f-25fc-4041-ba10-f85b02412580 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.588199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67fc3b-cbfb-4844-9a3e-2283ae708d10 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.590922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.591786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.592256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.592716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.630923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.631558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.631831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.632095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.632668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.634088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:17.645916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:17.646905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 423/843] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:17.647752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:17.655205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.672562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.675375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.780793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bdbbf-70f6-44e7-a272-fcf8a37093c4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.786967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:17.799093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 421/844] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:18:17.804800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.828753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.829365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.830150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.830557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.831481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:17.831778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.833306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.842805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 424/845] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:17.843861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:17.909383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:17.922712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558b128-3d87-4d0d-bfad-3a2f98cc4933 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.960666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:17.961311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.961585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.961858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.964751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab3d7b-dc1a-42f9-b9b5-40835635af86 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:17.973139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:17.974578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-3b16197c-1040-4ddd-875c-03d822c0b9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:17.990050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:17.990982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:17.991863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:17.991863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:17.991863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:17.991863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 422/846] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:17.992356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2434cb-8da3-49a8-acc7-80246a3c567e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:17.992559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2434cb-8da3-49a8-acc7-80246a3c567e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:17.992857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2434cb-8da3-49a8-acc7-80246a3c567e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:17.994210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 423/847] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:17.996473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2434cb-8da3-49a8-acc7-80246a3c567e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.010535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.011140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.011418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.011686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.012311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:18.012608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.014016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c7778-5b3a-4bca-811c-c3f7a05b1d60 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:18.019459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 425/848] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:18.029070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.029070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.029070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.051075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:18.053245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:18.060199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:18.130374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2434cb-8da3-49a8-acc7-80246a3c567e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.162223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 426/849] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:18.182940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.183478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.183753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.184024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.184670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:18.184999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.186578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e703c-116a-4a20-89ff-7c6360eb5920 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:18.200171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 424/850] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:18.206071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.257063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:18.262075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51786e-4890-4e0f-a09b-feffcb4a7984 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.262362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51786e-4890-4e0f-a09b-feffcb4a7984 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.262646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51786e-4890-4e0f-a09b-feffcb4a7984 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.264114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 425/851] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:18.269229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f173-0c65-41dd-988b-524941f663be None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.269508 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f173-0c65-41dd-988b-524941f663be None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.269788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f173-0c65-41dd-988b-524941f663be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.291785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-7b96f173-0c65-41dd-988b-524941f663be None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:18.293530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7b96f173-0c65-41dd-988b-524941f663be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:18.321273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65eb8-7fba-4ab6-a393-e74619182a47 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.364864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.366172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.366619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.367032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.374682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.378351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:18.390256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:18.391374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:18.392804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:18.393529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 426/852] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:18.395153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 427/853] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:18.400990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f173-0c65-41dd-988b-524941f663be None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.438091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:18.440305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.516309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f173-0c65-41dd-988b-524941f663be None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.555057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.555850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.556226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.556701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.557642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:18.558070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.559662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:18.560705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e863a-2660-4d61-a77d-8e0b61f2d417 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.564157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:18.580170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 427/854] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:18.581696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.618666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.619535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.620213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.620633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.634591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:18.640125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:18.641077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:18:18.641921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:18.644082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 428/855] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:18.647827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.707570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:18.718623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464081-3fad-4d83-99b8-dab499a7bc2b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.756101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.756693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.756998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.757270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.758013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=9d8eb54303a94ec38dbb9209c71381da) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:18.758315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.759736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:18.765982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 428/856] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/role_assignments?user.id=9d8eb54303a94ec38dbb9209c71381da&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:18.768086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0103dd-3ce9-435f-9a99-b60716e5c57e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.773830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.808511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.809179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.809462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.809847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.810400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:18.810752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.812144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:18.827426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 429/857] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:18.828805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:18.861681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:18.942687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d601bd-e5ba-452e-bc46-3b79d9a3ab96 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.974563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0d177-bf42-43c1-95d9-b75309207f65 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:18.988889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:18.989529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:18.989812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:18.990069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:18.990660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:18.992135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:19.002378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:19.003361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:19.004256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:19.006797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 429/858] 10.222.0.81 () {56 vars in 1132 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:19.013341 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:19.024142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.025109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.025605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.026075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.038608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.040512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.045772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:19.058625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:19.059643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:19.060526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:19.060526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:19.060526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:19.062609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 430/859] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:19.070164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:19.139382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b292dd-00eb-45db-99af-c6784e825c5d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:19.178306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.178942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.179231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.179482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.180130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:19.180428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.181864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.182899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:19.187789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 430/860] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:19.193539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:19.219086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b1098-6a06-41c4-a00e-2904461fa505 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:19.258875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.259561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.259938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.260307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.261085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:19.261481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.263178 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.281953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 431/861] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:19.284022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:19.300010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:19.321659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081aadb-1afb-43e2-86fd-6c1b2b2eb70f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:19.371840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.372876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.373349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.373798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.374616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:19.375094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.377487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.389154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 431/862] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:19.410227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19ab7f-084b-4830-972d-c7c46f7767bc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:19.449878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.450531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.450806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.451075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.460445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.461965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.464237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:19.476610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:19.478099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:19.479301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:19.481933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 432/863] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:18:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:19.488041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:19.619540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9419c0-1105-4624-b787-a252d7a0b0ab None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:19.658549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.659540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.660020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.660508 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.661464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:19.661982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.663759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619315e-f24a-4ce8-9be2-ea7f68a44d37 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:19.685852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 432/864] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Aug 24 03:18:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:19.694590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:19.816123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf22f56-1f6b-4379-aa8f-47c3aa16b9d2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:19.853221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:19.854096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:19.854515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:19.854925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:19.928405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:19.932362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:19.933876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.035133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fa5e3-eb68-4e2e-8a23-f201d0a7adbc None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.035641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fa5e3-eb68-4e2e-8a23-f201d0a7adbc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.035641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fa5e3-eb68-4e2e-8a23-f201d0a7adbc None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.037269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 433/865] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:20.042109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.042387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.042655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.044252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 433/866] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:18:20.050390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:20.061046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:20.062475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:20.081049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:20.147604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 434/867] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:20.165167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d77cc0-6f23-40ab-9434-8b31359bc6c5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:20.197942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:20.202491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:20.203398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.203875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.204343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.216457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:20.218175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:20.234861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:20.235856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:20.237264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:20.237264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:20.237264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:20.239403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 434/868] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:20.244829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:20.350116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3a5b4-9b51-49de-8c77-f5f61486590b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:20.386323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:20.386936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.387219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.387494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.388142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:20.388441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:20.389836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.397407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:20.407272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 435/869] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:20.408453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:20.420751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:20.540065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcd0ed-07c2-4339-92f8-021d5c4c53a0 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:20.542530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82372c-5982-4eae-ba64-c527bc5d8cb7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:20.580529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:20.581614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.582093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.582580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.583442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:20.585027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.586248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:20.586876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.587154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.587425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.596758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:20.598036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.606282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 436/870] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:20 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:20.620873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:20.621855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:20.622736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 435/871] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:20.627283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:20.702980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:20.759590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c462ce5-68ea-44da-b049-367b4ba3922e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:20.795017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:20.795641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.795920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.796188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.796932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:20.797233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:20.799293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9453397-ecf9-4522-a233-3131d05b7506 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.812492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 437/872] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:20.818709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:20.834648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:20.846877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672eed1-f2d4-4a67-a177-c4b78f44752e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.847174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672eed1-f2d4-4a67-a177-c4b78f44752e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.847437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672eed1-f2d4-4a67-a177-c4b78f44752e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.848960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 438/873] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:20.854667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522986be-e8a4-42db-b571-037187f55065 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.855185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522986be-e8a4-42db-b571-037187f55065 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.855658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522986be-e8a4-42db-b571-037187f55065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.877793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-522986be-e8a4-42db-b571-037187f55065 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:20.879009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-522986be-e8a4-42db-b571-037187f55065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:20.947574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c8fbe-e801-4dc3-a92e-732ecaf11729 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:20.973061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 439/874] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:20.984611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:20.985266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:20.985541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:20.985845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:20.986838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=cbd35279778f4858808385011249d9f5) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:20.987156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:20.988595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:20.995104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 436/875] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:20 2022] GET /identity/v3/role_assignments?user.id=cbd35279778f4858808385011249d9f5&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:21.242888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522986be-e8a4-42db-b571-037187f55065 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:21.291617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:21.349591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522986be-e8a4-42db-b571-037187f55065 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:21.386751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:21.387524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:21.387901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:21.388273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:21.388998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:21.390482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-4c7de80a-98ae-4578-869e-18995537c1b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:21.400141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:21.401382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:21.402424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:21.404448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 440/876] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:21.410545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:21.490492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:21.508615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564280c-e66a-41ef-ab1e-120bd5564edb None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:21.508839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564280c-e66a-41ef-ab1e-120bd5564edb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:21.509119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564280c-e66a-41ef-ab1e-120bd5564edb None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:21.510806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 441/877] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:21.515618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:21.515898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:21.516166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:21.523213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025fbb2-e2fb-4118-b3d1-39bf85b08b7b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:21.536487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:21.538282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:21.567242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:21.569006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:21.569615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:21.570074 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:21.571548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:21.572159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:21.574100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:21.580246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 437/878] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:21.587969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:21.628879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 442/879] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:21.697030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9c4f2-1759-4cdc-bd91-94225867bbe5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:21.733134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:21.733742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:21.734022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:21.734318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:21.734977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:21.735299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:21.737574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2311e-82d2-4f33-9168-fcaa349219d4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:21.742802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 438/880] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:21.908240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:21.951942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:22.017425 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6f155-b9d1-4baf-898c-24133d00bfb1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:22.054472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd0368-d448-4e0e-9594-2d13e9c9e606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:22.055074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd0368-d448-4e0e-9594-2d13e9c9e606 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:22.055348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd0368-d448-4e0e-9594-2d13e9c9e606 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:22.055620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd0368-d448-4e0e-9594-2d13e9c9e606 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:22.056277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd0368-d448-4e0e-9594-2d13e9c9e606 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:22.057731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd0368-d448-4e0e-9594-2d13e9c9e606 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:22.080630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 443/881] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:21 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:22.740013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114aa62f-4e1b-4847-b531-2ffc3f3a4045 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:22.740560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114aa62f-4e1b-4847-b531-2ffc3f3a4045 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:22.740560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114aa62f-4e1b-4847-b531-2ffc3f3a4045 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:22.742199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 439/882] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:22.748018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c478bc-8ad6-499b-8afa-5cf47cbdd475 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:22.748468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c478bc-8ad6-499b-8afa-5cf47cbdd475 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:22.748645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c478bc-8ad6-499b-8afa-5cf47cbdd475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:22.767528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f7c478bc-8ad6-499b-8afa-5cf47cbdd475 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:22.768855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f7c478bc-8ad6-499b-8afa-5cf47cbdd475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:22.847230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 444/883] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:23.073259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1139-db1b-4a57-9582-9045bb4e8ec5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.073672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1139-db1b-4a57-9582-9045bb4e8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.074076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1139-db1b-4a57-9582-9045bb4e8ec5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.075773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 440/884] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:23.081763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.082171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.082544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.101846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:23.103312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:23.122008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:23.127508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e1139-db1b-4a57-9582-9045bb4e8ec5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.130479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:23.177363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 445/885] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:23.244636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e1139-db1b-4a57-9582-9045bb4e8ec5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.280634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.281294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.281569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.281840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.282419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:23.284936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:23.293989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:23.294967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:23.296486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 441/886] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:23.302515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.407189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e72639-a8ba-41a8-8f50-a607b7aa3a15 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.444643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.445533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.445957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.446374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.447174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:23.447616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:23.449205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:23.454407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 446/887] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:23.460634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.496565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.526665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:23.541320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:23.583788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba306a-854a-4d42-ae4f-cb608cd469f3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.636982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b6d43-616f-4de4-bdc3-82b98109c166 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.637459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.638068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.638343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.638613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.639245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:23.639542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:23.643071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:23.652838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 442/888] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:23.654881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.690754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.691304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.691581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.691851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.706334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:23.712433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:23.713418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:23.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 447/889] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:23.722128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.770368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e8065-2e2e-4718-836a-9e2b9def447b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.802913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:23.809800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.810430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.810704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.810998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.811693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] RBAC: Adding query filter params (interface=public, service_id=d8b5fd28b4ce47b4a878569ef013ad0f, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:23.811997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:23.813409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9464c9-335a-414a-90a1-e5297c135385 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:23.823629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 443/890] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/endpoints?service_id=d8b5fd28b4ce47b4a878569ef013ad0f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:23.823629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec799e9-3ab7-4168-b97c-0fc50088729d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.823629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec799e9-3ab7-4168-b97c-0fc50088729d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.823629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec799e9-3ab7-4168-b97c-0fc50088729d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.823629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 444/891] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:23.824645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec799e9-3ab7-4168-b97c-0fc50088729d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:23.829311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:23.853159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3054238-aebf-48f1-96c9-1b35bdba6bfe None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.889441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.890064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.890345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.890613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.891271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:23.891573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:23.893063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea203-9c4e-49f2-94b3-9c0c0d062991 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:23.901462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 448/892] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:23.902463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d736-f9f0-4566-96a0-4b32b1935e45 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.902743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d736-f9f0-4566-96a0-4b32b1935e45 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.903010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9d736-f9f0-4566-96a0-4b32b1935e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.929706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9db9d736-f9f0-4566-96a0-4b32b1935e45 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:23.930895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9db9d736-f9f0-4566-96a0-4b32b1935e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:23.947283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec799e9-3ab7-4168-b97c-0fc50088729d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:23.987364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:23.988005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:23.988280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:23.988552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:23.989189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:23.990566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:23.996332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 445/893] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:18:23.997141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.031008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 449/894] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:24.074915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a77ce-6480-4aa8-ad64-3566537a3b4e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.075200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a77ce-6480-4aa8-ad64-3566537a3b4e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.075480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a77ce-6480-4aa8-ad64-3566537a3b4e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.081737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 450/895] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:24.083014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:24.083593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.083874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.084152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.103852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:24.105128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:24.132388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145fc42-d434-4775-abd2-688ec94626a5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.174774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.175807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.176352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.176964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.190607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.192975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.205767 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:24.210579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:24.211705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:24.212666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:24.212666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:24.212666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:24.213490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 451/896] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:24.214527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 446/897] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:24.220678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.315843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.331136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda1e53-97f0-4bde-96c6-885c3034ea9d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.368809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.369428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.369707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.369982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.370670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:24.371073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.372678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.385178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 452/898] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:24.390632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.439581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baea226-c398-4f5c-a613-4d9c5f05faed None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.453075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:24.469220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:24.487047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.487658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.487933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.488203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.488873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.490247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.506677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 447/899] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:24 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:24.507754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.530772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9acaa-e747-4dd0-b88c-5a307e227c76 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.591213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.592394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.592868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.593575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.605893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.607933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:24.625229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:24.626442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:24.627601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:24.630869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 453/900] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:24.636840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61ffac-7a72-4287-8490-bff04b6411bb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.637035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.673053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.674390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.674687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.674957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.675497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.676900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:24.686639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:24.687642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:24.688564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 448/901] 10.222.0.81 () {56 vars in 1132 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:24.693709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.741956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:24.778451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685e86-8d7c-4ac4-bdf6-8703f990cdd5 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.798665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:24.808483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c244bbc-f0eb-4020-a358-be492b214c2c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:24.826518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.827384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.827729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.828526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.829666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:24.830557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.832260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.853305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:24.854257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:24.854598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:24.854929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:24.855624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:24.855988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:24.857502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:24.860737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 454/902] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:24.862760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:24.868871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 449/903] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:24.871428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.003385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:25.007406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ba10-0a16-4dbc-9b14-d1b36424368a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.013621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21399ab2-28f3-4024-b186-914c245237b5 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.046868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.047949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.048522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.049025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.049845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:25.050368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.051932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.053357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.054136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.054414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.054689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.063285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 450/904] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:25.065130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.110818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:25.144285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.145870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.182699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5906fed-460c-497b-be4a-0141e0fa6106 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.221760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.222465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.222844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.223187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.224050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] RBAC: Adding query filter params (interface=public, service_id=f18e2e413ea940a6be19baf62d2db94f, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:25.224450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.226050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.237747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 451/905] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:25 2022] GET /identity/v3/endpoints?service_id=f18e2e413ea940a6be19baf62d2db94f&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:25.238795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.258480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 455/906] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:18:25.264069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.269281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:25.368818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:25.377537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949be1c-2c90-4800-8ee1-94993b73afbb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.402455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82387-799f-4f05-a7e9-e18f01216057 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.424898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.426703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.427212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.428220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.429478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.431483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.444859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 452/907] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:25 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:18:25.451335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.452394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.452925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.453437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.466358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.468319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:25.486683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:25.487752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:25.488651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:25.488651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:25.488651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:25.490961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 456/908] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:25.497290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.604798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:25.615226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839708d0-cb49-431e-8c48-ee0dec44e11f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.652787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.653416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.653694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.653969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.654614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:25.654990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.656460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.670484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 453/909] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:25.676901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.800326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d90fa-6727-43b6-be5c-44476ee08c08 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:25.828452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:25.836966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:25.837582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:25.837862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:25.838141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:25.845269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:25.847416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:25.859707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:25.860616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:25.862208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 457/910] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:25.869304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:25.980522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749365f-18ed-44cf-a7fb-fde76257f698 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:26.018162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:26.018875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.019150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.019420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.020169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:26.020470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:26.021952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa5d64-297c-4974-abe4-3b1caba9a658 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:26.034874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 454/911] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:26.041029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:26.139166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc488e7-94f1-46c8-bf74-6109cd7cf923 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.139489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc488e7-94f1-46c8-bf74-6109cd7cf923 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.139645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc488e7-94f1-46c8-bf74-6109cd7cf923 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.144804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 455/912] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:26.147429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fbb6e-5f1f-4f1d-b849-623505e6a256 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.147927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fbb6e-5f1f-4f1d-b849-623505e6a256 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.148421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fbb6e-5f1f-4f1d-b849-623505e6a256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.155195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6429b-dda1-4572-b660-911cfa571dfa None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:26.169261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-993fbb6e-5f1f-4f1d-b849-623505e6a256 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:26.170877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-993fbb6e-5f1f-4f1d-b849-623505e6a256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:26.187098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:26.191157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:26.191786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.192061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.192569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.193357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] RBAC: Adding query filter params (role.id=4b669e9fe0fa42e59073f7443dadf4b8, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=cbd35279778f4858808385011249d9f5) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:26.193664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:26.195186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:26.201709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 458/913] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:26 2022] GET /identity/v3/role_assignments?user.id=cbd35279778f4858808385011249d9f5&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=4b669e9fe0fa42e59073f7443dadf4b8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:26.251987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 456/914] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:26.543933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:26.607989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:26.665281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470a361-9fed-415e-b709-65066d10d275 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:26.687976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4ac0-e753-49ef-a825-a573e1900bfe None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.688313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4ac0-e753-49ef-a825-a573e1900bfe None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.688533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4ac0-e753-49ef-a825-a573e1900bfe None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.690314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 457/915] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:26.698868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.699152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.699425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.712804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:26.713443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.713721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.713997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.714589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:26.715960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:26.726994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:26.728307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:18:26.729179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:26.731988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 459/916] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:26.738830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:26.755543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:26.756750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:26.895630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75304df-d010-4a4a-87dd-d9901943f6b5 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:26.920874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 458/917] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:26.921355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:26.945682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:26.946456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:26.946827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:26.947200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:26.947941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:26.948359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:26.949923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:26.955424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 460/918] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:26.963372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.044999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:27.078580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ba8ca-f6c7-4832-9ce1-481767f018b4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:27.133002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:27.133615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.133889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.134160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.134790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:27.135090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:27.136568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:27.142213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 459/919] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Aug 24 03:18:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:27.151034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.262967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267268c-9805-486b-bd08-e5b0740672f6 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:27.266059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.299862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:27.300476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.300887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.301125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.301785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:27.303257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:27.326520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 461/920] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:27.388146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf8c8d-e74b-4438-8979-b6c88fa7ba14 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:27.433671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:27.436799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:27.437888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.438552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.439065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.440547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:27.442480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-30d5b401-da76-4795-9d15-61b3bcb4a706 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:27.453070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:27.454152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:27.455069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:27.458079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 460/921] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:27.466252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.507875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:27.553730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c4053-14f8-462e-b199-8b5029a5ba16 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.554044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c4053-14f8-462e-b199-8b5029a5ba16 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.554289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c4053-14f8-462e-b199-8b5029a5ba16 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.556255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 461/922] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:27.561841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.562120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.562393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.583181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59967683-3ece-4072-9d33-aa2106eb2fd7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:27.586094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:27.587292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:27.627876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:27.629093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.629574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.630902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.631738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:27.632246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:27.633843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:27.641128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 462/923] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:27.648065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.674788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 462/924] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:27.755884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:27.760585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab678e0-38fe-4c96-a4ec-5e269f270b2b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:27.797655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:27.798679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.799663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.800348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.801209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:27.801766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:27.803397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:27.809290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 463/925] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:27.816201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.937227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c039150-6d28-45df-af1b-31c503cd88eb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:27.945140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:27.963329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:27.975718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:27.976323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:27.976618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:27.976940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:27.977995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] RBAC: Adding query filter params (interface=public, service_id=6ecaf521aaf84ea78f24a56bb8d4f7fe, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:27.978298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:27.979790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:27.985936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 463/926] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:27 2022] GET /identity/v3/endpoints?service_id=6ecaf521aaf84ea78f24a56bb8d4f7fe&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:27.992716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:28.078243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355b9af-a6ff-4068-b2fe-6b4fda75dbd7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:28.107798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fc2b9-4d1e-4eef-9d5d-4fedde060e40 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:28.117605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:28.118515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.118995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.119537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.120322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:28.122017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:28.132727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:28.133687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:18:28.134597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:28.137557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 464/927] 10.222.0.81 () {56 vars in 1132 bytes} [Wed Aug 24 03:18:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:28.143516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:28.149897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:28.150775 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.151245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.151725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.152533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:28.154118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:28.166514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 464/928] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:27 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:18:28.182121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:28.263954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eacd47-dca3-48bc-a740-41570afa2738 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:28.301756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:28.302367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.302640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.302907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.303544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:28.303838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:28.305460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ca355-6e79-418c-a477-34ac60afcd28 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:28.316522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 465/929] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:28.318059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:28.320005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12646f-5bd6-4109-bddb-b8060d86e1ef None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.320289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12646f-5bd6-4109-bddb-b8060d86e1ef None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.320561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12646f-5bd6-4109-bddb-b8060d86e1ef None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.322185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 466/930] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:28.328418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.329028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.329534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.349520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:28.350727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:28.394160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:28.428690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 467/931] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:28.450944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacedc9-b181-4fdf-a307-c4cdc48cec7b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:28.488227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:28.488862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.489143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.489417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.490047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:28.490348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:28.491814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:28.497446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 465/932] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:28.504672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:28.526853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:28.619162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671f4e-f9db-433d-86f6-68ae31719f86 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:28.656724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:28.657348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.657623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.657888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.658593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:28.660084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:28.681990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 468/933] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:28.734906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:28.850761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f29fa-81f1-4e91-ba42-04d2a3db7d2f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:28.869551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:28.894750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:28.895383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:28.895660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:28.895931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:28.907136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:28.912961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:28.913873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:28.915111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 466/934] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Aug 24 03:18:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:28.920320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.003351 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:29.034194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f28f1-75a3-4c0e-8029-27cb5690384e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:29.072431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:29.073066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.073344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.073613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.074285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:29.074587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:29.076652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:29.085476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 469/935] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:18:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:29.091192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.198427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcf530-b7af-447a-8901-9ff0973ec65b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:29.242355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:29.243055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.243331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.243608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.255564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:29.257055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:29.271480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:29.272528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:29.273846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:29.273846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:29.273846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:29.273846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 467/936] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:29.278533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.390535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10d52-d27c-467b-8900-fa5cb154596f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:29.427778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:29.428370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.428640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.428951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.429588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:29.429893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:29.431228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02ab85-cfb7-445e-a235-48ec4f37dd61 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:29.442931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:29.445818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 470/937] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:29.454136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.479462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92dca5-468e-4f84-9d8e-a63cfe02601f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.479770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92dca5-468e-4f84-9d8e-a63cfe02601f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.480732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92dca5-468e-4f84-9d8e-a63cfe02601f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.481566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 471/938] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:29.484467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:29.489897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.490422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.490911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.516898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:29.519260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:29.576087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad266f-9c64-44dc-b4f4-e5644cc0d83d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:29.609464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 472/939] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:29.618412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:29.619344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.619831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.620291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.629322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:29.631076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:29.642382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:29.644283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:29.645258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:29.647696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 468/940] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:18:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:29.655771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.762199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b5d20-ccf6-43f1-88e7-c2bbf47c63c8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:29.790343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:29.798544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:29.799299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:29.799658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:29.799876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:29.800615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:29.801350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:29.802787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:29.825145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 473/941] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Aug 24 03:18:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:29.831399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.898911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:29.934890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:29.958638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62143-00b0-43b5-b94f-cf6557dc6864 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.000932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.001555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.001909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.003508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.014624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.016126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:30.032479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:30.034211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:30.035130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:30.035130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:30.035130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:30.039064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 469/942] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:30.040003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.040999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c5e0-5541-4951-928f-19a86e4dcf1c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.082865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.082865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.083154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.083403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.084152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.085647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.088414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING keystone.server.flask.common [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 03:18:30.107348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.sql.core [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:18:30.107348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 03:18:30.107348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [parameters: {'id': 'd20ac48e48594ab990efa778fae33f07', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 03:18:30.107348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81486) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d20ac48e48594ab990efa778fae33f07.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d20ac48e48594ab990efa778fae33f07. Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:18:30.127217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application result.read() Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 03:18:30.128263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application yield resource Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise self.value Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application session.commit() Aug 24 03:18:30.129499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:18:30.130583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:18:30.131646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application result.read() Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:18:30.132616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [parameters: {'id': 'd20ac48e48594ab990efa778fae33f07', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.139897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:30.141770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d20ac48e48594ab990efa778fae33f07. Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 474/943] 10.222.0.81 () {60 vars in 1140 bytes} [Wed Aug 24 03:18:29 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:30.144019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.174542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7a834-44f8-4b0a-8547-f5811a283c13 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.193038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:30.218429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91091de3-390b-4b52-b306-514e587f793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.219280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91091de3-390b-4b52-b306-514e587f793d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.219696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91091de3-390b-4b52-b306-514e587f793d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.220103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91091de3-390b-4b52-b306-514e587f793d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.222184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91091de3-390b-4b52-b306-514e587f793d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:30.222812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91091de3-390b-4b52-b306-514e587f793d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.225145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91091de3-390b-4b52-b306-514e587f793d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.229027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:30.244803 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 470/944] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:30.249906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91091de3-390b-4b52-b306-514e587f793d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.272589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d506b-490c-46c2-a8cc-54f7a96d257c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.307989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.309127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.309731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.310222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.318326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.320229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-92174663-265c-4c4d-b58d-8f551afea87e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:30.334563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:30.335535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:30.336449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 475/945] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:30.337488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd4a9b-d0c1-4328-b7d2-f9327a0b84e7 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.337488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd4a9b-d0c1-4328-b7d2-f9327a0b84e7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.337623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd4a9b-d0c1-4328-b7d2-f9327a0b84e7 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.345229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 476/946] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:30.345229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd4a9b-d0c1-4328-b7d2-f9327a0b84e7 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.369752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91091de3-390b-4b52-b306-514e587f793d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.406572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.407478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.407908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.408224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.415300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.416751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7d499ad8-ffe3-47b2-8045-ff7361764ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:30.428885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:30.429783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:30.430699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:30.434593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 471/947] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:30.434593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.435096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.435373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.454027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:30.455369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:30.464682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd4a9b-d0c1-4328-b7d2-f9327a0b84e7 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.467076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:30.505678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.506276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.506552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.506819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.507476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:30.507774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.509233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.512123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:30.528198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 477/948] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:30.530757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 472/949] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:30.536266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.657394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e905f-ff0f-462a-86dc-3cb2e5cda9e9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.696817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.697812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.698284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.698746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.699640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:30.700132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.701780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900d4a-05ad-41a2-9d26-848463dadfdd None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.714735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 478/950] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:30.720423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.734574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8612f9-3512-40a8-ad48-d4ddea22eb6f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.734878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8612f9-3512-40a8-ad48-d4ddea22eb6f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.735161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8612f9-3512-40a8-ad48-d4ddea22eb6f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.737426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 479/951] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:30.742991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.743277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.743556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.769821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:30.771396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:30.792897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:30.838286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1dfe-6781-42b8-8ae0-590ff33c85c0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:30.860791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 480/952] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:30.860791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:30.892253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:30.893675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:30.894040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:30.894319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/cbd35279778f4858808385011249d9f5/roles/4b669e9fe0fa42e59073f7443dadf4b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:30.896790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=cbd35279778f4858808385011249d9f5, role_id=4b669e9fe0fa42e59073f7443dadf4b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:30.898986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:30.925335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 473/953] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:18:30 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/cbd35279778f4858808385011249d9f5/roles/4b669e9fe0fa42e59073f7443dadf4b8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:18:30.992264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10f9df-f372-4404-bfe8-438926758f39 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.031644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.032636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.033091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.034035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.034808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.036487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:31.046594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:31.048283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:31.050203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:31.052993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 481/954] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:31.059376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:31.120094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:31.137551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:31.165138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:31.186706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf546d-488f-4758-b299-52b2b919daf5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.233448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.234210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.234602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.234879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.235511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:31.235809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.238020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.248840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 474/955] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:31.252758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:31.288172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e11f5a-77b4-4f65-a8bf-011223d9d107 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.336860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.337465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.337749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.338028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.338880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.340987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:31.352561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:31.354003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:31.354873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:31.359616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 482/956] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:31.360616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:31.396160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc142586-859d-415a-b432-49f1af131a0c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.410296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:31.442126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.443369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.443741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.444127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.445104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:31.445469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.447828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.456950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 475/957] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:31.462572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:31.464336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:31.488128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec759da-4297-411c-8b79-2c2314778167 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.530760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.531446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.532216 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.532558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.533838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:31.534312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.535967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.544844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 483/958] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:31.549631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:31.597454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20dda7-2950-40e7-b4a4-ef5606b87cd2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.639084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a9ad4-e8ec-485a-b57a-9ffed633e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.639697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a9ad4-e8ec-485a-b57a-9ffed633e2cb None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.639974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a9ad4-e8ec-485a-b57a-9ffed633e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.640242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a9ad4-e8ec-485a-b57a-9ffed633e2cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.640930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a9ad4-e8ec-485a-b57a-9ffed633e2cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.642304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a9ad4-e8ec-485a-b57a-9ffed633e2cb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.668319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 476/959] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:18:31.698481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbe32b-0e3f-43ed-bb35-0f46fe5d0f5d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:31.756079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:31.756706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:31.757336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:31.757701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:31.758862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:31.759412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:31.761563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6804d-060d-44e7-af3b-fce56c9c5108 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:31.769476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 484/960] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:32.708023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08186b-0d11-4f4d-bc19-843e4ac92e1f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:32.708543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08186b-0d11-4f4d-bc19-843e4ac92e1f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:32.708893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08186b-0d11-4f4d-bc19-843e4ac92e1f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:32.710603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 477/961] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:32.715989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ddfd8-6eab-46aa-b6c8-4027214af104 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:32.716408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ddfd8-6eab-46aa-b6c8-4027214af104 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:32.716753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ddfd8-6eab-46aa-b6c8-4027214af104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:32.724324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:32.738346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-763ddfd8-6eab-46aa-b6c8-4027214af104 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:32.739907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-763ddfd8-6eab-46aa-b6c8-4027214af104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:32.813805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 485/962] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:32.944223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2684834-5cf5-4135-8171-cc9771cc1962 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:32.944746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2684834-5cf5-4135-8171-cc9771cc1962 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:32.945280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2684834-5cf5-4135-8171-cc9771cc1962 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:32.947075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 478/963] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:32.954248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:32.954756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:32.955313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:32.974145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:32.975659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:33.050791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 486/964] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:33.108283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2684834-5cf5-4135-8171-cc9771cc1962 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:33.142054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:33.221905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2684834-5cf5-4135-8171-cc9771cc1962 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:33.261093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:33.261618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.261890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.262159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.262783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:33.264199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:33.266741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING keystone.server.flask.common [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 03:18:33.279167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.sql.core [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:18:33.279167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 03:18:33.279167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [parameters: {'user_id': '3bf740adac524fe78b01b110f46a4918', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 03:18:33.279167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81485) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Aug 24 03:18:33.289569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:18:33.291037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application yield resource Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise self.value Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application session.commit() Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:18:33.291998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:18:33.292932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:18:33.295546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:18:33.297638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [parameters: {'user_id': '3bf740adac524fe78b01b110f46a4918', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:33.300487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 03:18:33.311036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:33.312679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 479/965] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Aug 24 03:18:33 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:33.313488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:33.368033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:33.386381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:33.434338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301aa50-23e4-4f73-b701-24fbb68da75f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:33.470910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:33.472245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.472629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.473378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.481330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:33.482851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:33.492611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:33.498554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:33.499886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:18:33.501061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:33.509006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 487/966] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:33.511434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:33.520086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e73869-d460-4b43-8394-a47ed68730de None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:33.558967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:33.559743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.560122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.560513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.572906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:33.577780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:33.578952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:33.579806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:33.585184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 480/967] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Aug 24 03:18:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:33.587112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:33.645954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d26129-a94f-4ce4-b9ab-f0a03aeefb55 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:33.678981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:33.691674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:33.692373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.692745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.693127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.693917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:33.694289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:33.695795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a8a6-3d9b-4fed-9b6a-4bec95835e2e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:33.716612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f23666-8e15-4d52-8300-3cd5154dd030 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:33.734003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 488/968] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Aug 24 03:18:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:33.735400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2875b6-72f0-4699-847a-9e124de55ba9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.735778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2875b6-72f0-4699-847a-9e124de55ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.736123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2875b6-72f0-4699-847a-9e124de55ba9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.738250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 489/969] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:33.744140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79b26-d3c7-4208-841a-8e792d3ab6ea None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.744417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79b26-d3c7-4208-841a-8e792d3ab6ea None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.744692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79b26-d3c7-4208-841a-8e792d3ab6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.765208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b6f79b26-d3c7-4208-841a-8e792d3ab6ea None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:33.765892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:33.766346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b6f79b26-d3c7-4208-841a-8e792d3ab6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:33.766956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.767387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.767829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.769441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:33.770959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:33.772786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:33.792806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 481/970] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:18:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:33.797495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:33.837164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:33.878123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 490/971] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:33.899773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef591c-4c6f-4f28-b6f7-751c6edf255f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.900063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef591c-4c6f-4f28-b6f7-751c6edf255f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.900343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef591c-4c6f-4f28-b6f7-751c6edf255f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.901905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 491/972] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:33.912458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.913000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.913461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.926286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00475ad0-bd25-4be2-b91b-5d6da369d7aa None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:33.936804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:33.937832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:33.969370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:33.970116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:33.970395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:33.970662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:33.983636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:33.985158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:34.001897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:34.002947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:34.003852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:34.003852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:34.003852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:34.003852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 482/973] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:34.009839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.037588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 492/974] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:34.090453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.128716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837c411-bcf8-44ab-b5dd-d64ea9964580 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.150813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.167640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.168511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.168808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.169078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.169880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:34.170480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.171935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.191332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 483/975] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:34.192459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.197691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.264987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0a06-f68c-4a20-97ac-d76079307b4b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.299850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcadf91-e9e9-42e4-aa14-2c38a6a6ae88 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.302682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.303301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.303572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.303838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.304412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.305954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:34.316343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:34.317443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:34.318296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 493/976] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:34.319108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.332964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.342425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.343048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.343322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.343590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.351124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.352568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:34.365066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:34.366016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 484/977] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:34.366954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.426316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eceb6e-09d9-474a-b71a-eb16de1d188a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.433219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.461748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.462348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.462624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.462908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.463549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.464942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.468380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153582-ae95-481d-a7ac-1dd3db9e1a15 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.479438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 494/978] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:34 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:34.480225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.518614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.519842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.520489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.520799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.521480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:34.521786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.524646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.543610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 485/979] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:34.544841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.602081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32179a-5da9-4860-af5c-83f1756f98ae None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.610509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.642326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.642968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.643241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.643657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.644402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:34.645460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.646881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.666161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 495/980] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:34.672297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.678139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a323-5090-48a3-8103-0ee0ef051e08 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.698971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.719865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.720537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.720826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.721880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.723243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:34.723714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.725505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.737256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 486/981] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:34.739756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.776640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b6a4-9d39-4fb3-89ec-4f1aceb0b7a7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.812909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.813521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.813803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.814077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.857756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf92ac-1323-4467-bcb1-34433a412946 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:34.870128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:34.888309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.892688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.903822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:34.904452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:34.904835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:34.905109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:34.905810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] RBAC: Adding query filter params (interface=admin, service_id=6ecaf521aaf84ea78f24a56bb8d4f7fe, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:34.906116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:34.907484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:34.914817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 487/982] 10.222.0.81 () {56 vars in 1249 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/endpoints?service_id=6ecaf521aaf84ea78f24a56bb8d4f7fe&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:34.920226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:34.928480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:35.009810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 496/983] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:18:35.014994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:35.037070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce003ec-a99a-411f-a0c4-62767bdf4eaf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:35.070856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:35.071448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:35.071716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:35.071985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:35.072577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:35.073958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:35.082480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 488/984] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:34 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:18:35.132649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9945af-5801-487b-b6b9-dc0247466230 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:35.161248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:35.173959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:35.175286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:35.175630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:35.175965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:35.187712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:35.189345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:35.206644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:35.207875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:35.213884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:35.213884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:35.213884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:35.213884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 497/985] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:35.215055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:35.260189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:35.326184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77ce37-6938-4269-9bc8-cae4dcaa95e5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:35.361136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:35.362390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:35.362716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:35.363051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:35.363752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:35.364116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:35.365578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:35.380817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 489/986] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:35.383163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:35.488224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94cccf-708f-41e9-8b8c-ff988d5a3c22 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:35.525433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:35.526206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:35.526581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:35.526953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:35.534039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:35.535590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:35.547692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:35.548816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:35.550355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:35.551597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 498/987] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:35.557307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:35.659440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:35.664077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395538-b111-4f74-a747-96f354c4f880 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:35.699242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:35.699921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:35.700263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:35.700601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:35.701380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:35.701748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:35.703232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8715c-9d7e-40d0-af66-bf61ec6c1d67 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:35.715656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 490/988] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:35.721579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:35.758726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:35.825289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4624c-4e4f-46e9-8c68-641798037e1e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:35.860159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:35.860751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:35.861052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:35.861321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:35.862141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] RBAC: Adding query filter params (role.id=4b669e9fe0fa42e59073f7443dadf4b8, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=cbd35279778f4858808385011249d9f5) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:35.862523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:35.863954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720f2bf-68e4-4848-9ba1-c6faa171bec1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:35.877030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 499/989] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:35 2022] GET /identity/v3/role_assignments?user.id=cbd35279778f4858808385011249d9f5&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=4b669e9fe0fa42e59073f7443dadf4b8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:36.442987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469154c-809e-4ddf-9081-e89bc61bd536 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:36.443545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469154c-809e-4ddf-9081-e89bc61bd536 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:36.443545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469154c-809e-4ddf-9081-e89bc61bd536 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:36.445132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 491/990] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:36.450289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:36.450573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:36.450845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:36.468448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:36.469643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:36.539883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 500/991] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:36.810109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469154c-809e-4ddf-9081-e89bc61bd536 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:36.924353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469154c-809e-4ddf-9081-e89bc61bd536 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:36.960430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:36.961085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:36.961364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:36.961631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:36.962204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:36.963572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0c72a459-5347-42d9-8d92-ea7c2dbb9fc2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:36.972798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:36.973868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:36.974715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 492/992] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:36.980428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.011556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:37.033419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ed60-f728-41a9-943c-0593644284dc None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.034246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ed60-f728-41a9-943c-0593644284dc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.034606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ed60-f728-41a9-943c-0593644284dc None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.036244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 493/993] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:37.041247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.041753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.042245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.062444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:37.064224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:37.081555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:37.091594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75410a-271e-4823-9af0-469a523e7ca3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.131111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.131707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.131984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.132267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.132930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:37.133227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.134592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.137585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 494/994] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:37.146984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 501/995] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:37.148517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.250008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f3172-39c2-40b9-b74a-be27329912ae None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.284754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.285378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.285654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.285925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.286551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] RBAC: Adding query filter params (type=compute) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:37.286848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.288220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.293365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 495/996] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:37.299134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.395042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:37.403050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b73b85-6cb3-4e69-b262-02e59e8f4bf5 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.412087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.433628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:37.450561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.451166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.451698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.451978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.453514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] RBAC: Adding query filter params (interface=public, service_id=a74781c3c89043cb900b562619338603, region_id=RegionOne) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:37.453830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.455227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.461229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 502/997] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:37 2022] GET /identity/v3/endpoints?service_id=a74781c3c89043cb900b562619338603&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:37.466599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.525839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca860f-1baa-4eff-9182-156d6d260bbf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.563127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.563740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.564010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.564290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.564878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.566946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.575052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e70cc-0f5a-4f6a-a85d-15ccd422879b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:37.575470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:37.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:37.577707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 496/998] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Aug 24 03:18:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:37.582958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.622183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.622783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.623058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.623328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.623938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.625343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.644983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 503/999] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:37 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:18:37.676532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:37.700203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3ba5c-2ef5-46a7-ba1d-c6f01ff398bc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.738440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.739063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.739341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.739612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.740254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:37.740555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.742027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910cf89-0b5d-415c-a457-a604e5d14803 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.753359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 497/1000] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:37.754261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.782311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:37.837138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42184ab-1592-4eec-9238-86d387c48b95 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.837497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42184ab-1592-4eec-9238-86d387c48b95 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.837686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42184ab-1592-4eec-9238-86d387c48b95 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.839032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 498/1001] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:37.843834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.844117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.844398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.861515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d5ce-ac68-4b97-bfb5-1e24842b3687 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:37.863294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:37.864497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:37.903249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:37.903858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:37.904137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:37.904410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:37.905087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:37.905394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:37.906768 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:37.915431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 504/1002] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Aug 24 03:18:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:37.920948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:37.950284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 499/1003] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:38.025713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:38.042598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98950b04-12a0-40fe-9fc5-f17bff8592bf None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:38.077226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:38.077826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:38.078102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:38.078373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:38.079030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:38.080402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:38.099973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 505/1004] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:18:38.234633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:38.314790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:38.348213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275dbee-89f0-4f1b-8f73-3293ae5f18b4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:38.399964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:38.400588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:38.400888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:38.401156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] PATH_INFO: `/v3/services/image` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:38.401725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:38.403081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:38.412212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:38.413260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:38.414118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 500/1005] 10.222.0.81 () {56 vars in 1105 bytes} [Wed Aug 24 03:18:38 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:38.420811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:38.523627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc501-2b1d-4254-b5d0-19759ff1d6fb None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:38.558681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:38.559279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:38.559553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:38.559822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:38.560445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] RBAC: Adding query filter params (name=image) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:38.560741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:38.562119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b55d29-ab34-4ab9-9029-1f0b5f6286ae None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:38.572164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 506/1006] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:38.573340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:38.691185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b19f69-2798-41ff-9426-914cf7d27496 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:38.726658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:38.727295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:38.727572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:38.727841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:38.728487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] RBAC: Adding query filter params (type=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:38.729076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:38.730512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032331e3-10eb-4a61-aba0-b0484b0adafd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:38.732094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:38.744799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 501/1007] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:39.570662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f528f-ac04-4fde-9c0c-b72f442abc26 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:39.571181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f528f-ac04-4fde-9c0c-b72f442abc26 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:39.571181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f528f-ac04-4fde-9c0c-b72f442abc26 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:39.572683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 507/1008] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:39.577883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096e9d7-31d7-4a62-b7ad-e8c74e9b6700 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:39.578313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096e9d7-31d7-4a62-b7ad-e8c74e9b6700 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:39.578457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096e9d7-31d7-4a62-b7ad-e8c74e9b6700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:39.597828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c096e9d7-31d7-4a62-b7ad-e8c74e9b6700 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:39.599029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c096e9d7-31d7-4a62-b7ad-e8c74e9b6700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:39.670173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 502/1009] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:39.937090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f528f-ac04-4fde-9c0c-b72f442abc26 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:39.950782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:40.043667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355928f-e07a-4dde-afd5-bfed762b2991 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.044014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355928f-e07a-4dde-afd5-bfed762b2991 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.044118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f528f-ac04-4fde-9c0c-b72f442abc26 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:40.044200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355928f-e07a-4dde-afd5-bfed762b2991 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.045576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 503/1010] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:40.050480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.050755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.051025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.069431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:40.070603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:40.079386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:40.080008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.080288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.080560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.081872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:40.083250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:40.092459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:40.093524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:40.094381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 508/1011] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:40.099802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:40.152931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 504/1012] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:40.181550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:40.207942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7a3d1-4120-4ede-b55e-6b37011770ad None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:40.244548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:40.245179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.245458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.245732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.246365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:40.246668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:40.248041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:40.253356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 509/1013] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:40.258868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:40.321529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:40.365946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b890d8-2dd1-476b-8f4d-be83e816f86c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:40.400378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:40.401007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.401286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.401556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.402183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:40.402489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:40.403885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:40.412801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 505/1014] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Aug 24 03:18:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:40.415347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:40.415547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:40.520077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adaf0e-cbc9-49cd-8270-95507610bb38 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:40.521135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54ea17-6557-4e42-b3af-aec65e7c0178 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:40.543456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:40.559884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:40.560524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.560821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.561008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ccbb4-d48c-4a46-be42-59db4f4fc2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:40.561119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.561612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ccbb4-d48c-4a46-be42-59db4f4fc2f9 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.561894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ccbb4-d48c-4a46-be42-59db4f4fc2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.562165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ccbb4-d48c-4a46-be42-59db4f4fc2f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.562267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:40.562825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ccbb4-d48c-4a46-be42-59db4f4fc2f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:40.563606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:40.564187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ccbb4-d48c-4a46-be42-59db4f4fc2f9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:40.568884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 506/1015] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:40.570679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:40.577014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:40.596947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 510/1016] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:40.664608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f8143-6768-42f5-8143-3b1979767dc9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.664966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f8143-6768-42f5-8143-3b1979767dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.665241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f8143-6768-42f5-8143-3b1979767dc9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.666722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 511/1017] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:40.672438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.672726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.673162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.686261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163d1b-0b6e-4535-a38f-8c4676d772e1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:40.693235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:40.694465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:40.734046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4b8ba-5a60-4915-a4e3-a01df7788204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:40.735459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4b8ba-5a60-4915-a4e3-a01df7788204 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:40.735459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4b8ba-5a60-4915-a4e3-a01df7788204 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:40.735459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4b8ba-5a60-4915-a4e3-a01df7788204 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:40.735970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4b8ba-5a60-4915-a4e3-a01df7788204 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:40.738180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4b8ba-5a60-4915-a4e3-a01df7788204 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:40.778381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 512/1018] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:40 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:40.780358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 507/1019] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:40 2022] POST /identity/v3/projects => generated 311 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:18:41.048170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:41.158008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59058f0-a987-4d65-bb1c-88f3ef9697b9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:41.182702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:41.197048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a0fc2-ac0a-44d1-976f-24283d099603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:41.197653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0fc2-ac0a-44d1-976f-24283d099603 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:41.197939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0fc2-ac0a-44d1-976f-24283d099603 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:41.198193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0fc2-ac0a-44d1-976f-24283d099603 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:41.198846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a0fc2-ac0a-44d1-976f-24283d099603 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:41.200206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a0fc2-ac0a-44d1-976f-24283d099603 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:41.216804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 513/1020] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:18:41 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:42.673839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3aeb-19f7-40f2-afe5-6f900a98b3e6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:42.674381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3aeb-19f7-40f2-afe5-6f900a98b3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:42.674381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3aeb-19f7-40f2-afe5-6f900a98b3e6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:42.676542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 508/1021] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:42.681644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:42.682038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:42.682188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:42.700803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:42.702012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:42.767266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb37cd-6318-4495-a31f-e5ece939a108 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:42.767545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb37cd-6318-4495-a31f-e5ece939a108 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:42.768036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb37cd-6318-4495-a31f-e5ece939a108 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:42.769797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 509/1022] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:42.775353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:42.775714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:42.776062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:42.779970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 514/1023] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:42.795706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:42.796965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:42.830095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:42.877123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 510/1024] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:43.051503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.147466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.155803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e7fcf-6d4a-41b1-b6c7-f00b32a67271 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.167923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.193589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c458d-abd4-404c-904e-3b24b8557e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.194211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c458d-abd4-404c-904e-3b24b8557e86 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.194489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c458d-abd4-404c-904e-3b24b8557e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.194761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c458d-abd4-404c-904e-3b24b8557e86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.204603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c458d-abd4-404c-904e-3b24b8557e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fd4c458d-abd4-404c-904e-3b24b8557e86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:43.210191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:43.211143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:43.211992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 515/1025] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:43.212676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44283a-76fe-4f1d-94dc-fa161585da9f None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.212753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44283a-76fe-4f1d-94dc-fa161585da9f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.213107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44283a-76fe-4f1d-94dc-fa161585da9f None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.218387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 516/1026] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:43.219029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44283a-76fe-4f1d-94dc-fa161585da9f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.253078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1d79e-1600-4f1b-9863-916068506ba5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.288376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9912d2-373c-4285-8e81-500cae644f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.289023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9912d2-373c-4285-8e81-500cae644f81 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.289300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9912d2-373c-4285-8e81-500cae644f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.289565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9912d2-373c-4285-8e81-500cae644f81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.290657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9912d2-373c-4285-8e81-500cae644f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.292115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9912d2-373c-4285-8e81-500cae644f81 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.293713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 511/1027] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:43.294804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.295077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.295344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.313241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:43.314446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:43.328018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44283a-76fe-4f1d-94dc-fa161585da9f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.344406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.364428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.365109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.365387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.365657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.366317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:43.366619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.367998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.376402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 517/1028] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:43.377188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.384233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 512/1029] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:43.385092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.416755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.492753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22c75-093f-4237-90f9-6424d7b9b77e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.495669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87d25d-9f57-4e78-8fa6-7a45bc8b82c8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.531278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.531666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.531892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.532151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.532339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.532439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.532621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.532924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.533098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.535086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.544376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.545966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.565204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 518/1030] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:18:43 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:43.565403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:43.566718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:43.567799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:43.567799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:43.567799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:43.567799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 513/1031] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:43.569215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.666023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.671220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.686807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcf504-67ac-4af9-ac8c-aa261fd7c630 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.688215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.727690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.728614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.729404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.729861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.730693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:43.731099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.732481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.745386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 519/1032] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:43.751117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.789381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cb6d8-8b9a-4a7f-89c0-41523e60f7e9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.825986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.826529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.826811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.827072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] PATH_INFO: `/v3/services/image` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.827662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.829057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:43.838754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:43.839770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:43.840619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 514/1033] 10.222.0.81 () {56 vars in 1105 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:43.845632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.869198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca62060-707b-4045-82ec-43bd7a15cf28 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:43.908843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:43.909659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:43.909922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:43.910279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:43.918135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:43.919582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:43.933380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:43.934590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:43.935536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 520/1034] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:43.939819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:43.945849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.951882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:43.967777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b3b4-0e4d-4c99-8b0d-761f41f749cc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.006705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.007310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.007587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.007860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.008490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] RBAC: Adding query filter params (name=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:44.008934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.010426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.020359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 515/1035] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:44.022253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.048732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefdcef-3206-4f52-8235-3cf6938516b7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.088115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.088694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.089006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.089278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.090009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:44.090312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.091792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.114015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 521/1036] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:18:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:44.116030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.137095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703db30-aa93-4469-bfa3-d8d383c6c78f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.173101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.173706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.173984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.174253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.174878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] RBAC: Adding query filter params (type=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:44.175205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.176566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.186288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 516/1037] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:44.187271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.195472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:44.202103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:44.235871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf0364-7675-4108-a77b-0d8fdbd9f3f9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.276286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.276926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.277208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.277481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.300451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf106a-5318-441e-a17a-86cc5a0e8b8b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.339788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.340293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.340571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.340877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.341575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] RBAC: Adding query filter params (interface=public, service_id=4dd5226068ba44f99c83e51b01b0a54e, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:44.341878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.343251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.348978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 517/1038] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/endpoints?service_id=4dd5226068ba44f99c83e51b01b0a54e&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:44.354632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.360887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.362619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.430494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:44.439540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:44.460294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed70ef-3080-4662-b22f-171930d90927 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.479890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 522/1039] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:18:44.485887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.496076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.496683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.496991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.497266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.497871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.499246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.506693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 518/1040] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:18:44.600643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be53b2-dec7-4881-87b1-ee7a7e5a2d77 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.639312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.639945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.640216 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.640483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.652095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.653583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:44.669529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:44.670486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:44.671367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:44.671367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:44.671367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:44.671367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 523/1041] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:44.677221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.783752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aff8d-26f3-4fff-a37a-749ec5f15bc1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.820585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.821220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.821501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.821769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:44.822401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:44.822696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:44.824065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:44.831623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:44.843368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 519/1042] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:44.846119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:44.870372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:44.959123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a400-b263-4c6d-91dd-cf02a5692302 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:44.995458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:44.996258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:44.996635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:44.997034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:45.004943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:45.006386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:45.017506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:45.018474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:45.020460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:45.024963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 524/1043] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:45.026163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:45.132565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d47e37-2100-4f1b-a416-8044226c4043 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:45.168751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:45.169387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:45.169662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:45.169931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:45.170623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:45.170924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:45.172352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190362-ca36-40dc-a08a-0ba15f72754b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:45.189467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 520/1044] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:45.191197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:45.273313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:45.296724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dca2b-9a43-4bc8-885e-c1d5b09c2914 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:45.333322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:45.333927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:45.334206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:45.334478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:45.335198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] RBAC: Adding query filter params (role.id=4b669e9fe0fa42e59073f7443dadf4b8, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=b8afb31c267946b7851cb1c22b62b6cb) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:45.335502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:45.336949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8b923-9a8f-4592-ad3e-3b0b601511a8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:45.348900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 525/1045] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:45 2022] GET /identity/v3/role_assignments?user.id=b8afb31c267946b7851cb1c22b62b6cb&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=4b669e9fe0fa42e59073f7443dadf4b8 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:45.546612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4905-0d13-4714-b6ac-4980870902f9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:45.546969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4905-0d13-4714-b6ac-4980870902f9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:45.547073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4905-0d13-4714-b6ac-4980870902f9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:45.548578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 521/1046] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:45.554302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909c677-c253-4e75-be78-40eccf52476f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:45.554586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909c677-c253-4e75-be78-40eccf52476f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:45.554860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909c677-c253-4e75-be78-40eccf52476f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:45.573648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f909c677-c253-4e75-be78-40eccf52476f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:45.574840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f909c677-c253-4e75-be78-40eccf52476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:45.648902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 526/1047] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:45.916722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf4905-0d13-4714-b6ac-4980870902f9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:45.980113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:46.024571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf4905-0d13-4714-b6ac-4980870902f9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:46.059011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:46.059624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.059894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.060159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.070108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:46.074641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:46.075561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:46.077697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 522/1048] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:46.081738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909c677-c253-4e75-be78-40eccf52476f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:46.151089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:46.184940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909c677-c253-4e75-be78-40eccf52476f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:46.219006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:46.219598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.219871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.220137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.220810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:46.221111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:46.223316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:46.236139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 527/1049] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:46.237368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:46.340732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25692664-d475-4960-ab2d-c55b011c0cdd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:46.359315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:46.377982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046699-6302-4244-90ea-25faefe97a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:46.378866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046699-6302-4244-90ea-25faefe97a3b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.379144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046699-6302-4244-90ea-25faefe97a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.379413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046699-6302-4244-90ea-25faefe97a3b None admin] PATH_INFO: `/v3/users/369de73754ec4717aefa6d021243a6e7` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.392938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78046699-6302-4244-90ea-25faefe97a3b None admin] RBAC: Authorizing `identity:get_user(user_id=369de73754ec4717aefa6d021243a6e7)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:46.395014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78046699-6302-4244-90ea-25faefe97a3b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:46.401577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 523/1050] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:18:46 2022] GET /identity/v3/users/369de73754ec4717aefa6d021243a6e7 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:46.402742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:46.489415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c65a01-2b90-413a-9792-8a7938b6dc5c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.489725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c65a01-2b90-413a-9792-8a7938b6dc5c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.490108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c65a01-2b90-413a-9792-8a7938b6dc5c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.491462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 524/1051] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:46.496335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.496614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.496935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.508931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdafc-30d2-4790-a1f9-9a4c365cc607 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:46.514451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:46.515622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:46.545730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:46.546541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.546820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.547087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.549788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:46.588890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 525/1052] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:46.619658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:46.621130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:46.727592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 528/1053] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:18:46.728959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:46.830503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33621d6c-deb4-4545-b68e-af40e8994bcf None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:46.838970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:46.862307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:46.868147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:46.868792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:46.869089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:46.869362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:46.870492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] RBAC: Authorizing `identity:get_project(project_id=af029992b398483096150f4f80112ff2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:46.871953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:46.873812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 526/1054] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:18:46 2022] GET /identity/v3/projects/af029992b398483096150f4f80112ff2 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:46.880664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:46.911457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:46.983664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544aacc-5c6f-42c6-9628-4a201474c228 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:46.993005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665345e-0f33-47cc-a1d1-32183857ff04 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:47.021590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.022218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.022499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.022780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.023370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:47.024816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:47.029034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.029634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.029908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.030175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.030887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=af029992b398483096150f4f80112ff2, user.id=369de73754ec4717aefa6d021243a6e7) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:47.031189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:47.032599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:47.033671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:47.034666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:47.035519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 529/1055] 10.222.0.81 () {56 vars in 1105 bytes} [Wed Aug 24 03:18:46 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:47.040797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 527/1056] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:46 2022] GET /identity/v3/role_assignments?user.id=369de73754ec4717aefa6d021243a6e7&scope.project.id=af029992b398483096150f4f80112ff2&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:47.049166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:47.160453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:47.166376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca584dd-5fb0-4e92-b1c1-e0605fe01eb3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:47.205642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.206243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.206996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.207339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.208175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] RBAC: Adding query filter params (name=image) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:47.208469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:47.209873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d3b5d-14b7-46cd-82f5-b27f899d9448 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:47.219627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 530/1057] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:47.221252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:47.256174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a83f5f-6a60-450d-b760-5d32ee056072 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.256430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a83f5f-6a60-450d-b760-5d32ee056072 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.256711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a83f5f-6a60-450d-b760-5d32ee056072 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.258263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 531/1058] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:18:47.262983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.263269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.263547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.265583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:47.282942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:47.284147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:47.335093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1945-27be-4122-b35a-05522e3d382d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:47.358709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 532/1059] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:47.370002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.370609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.370885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.371156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.371783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] RBAC: Adding query filter params (type=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:47.372081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:47.373485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:47.379078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 528/1060] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:47.384716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:47.487175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:47.496985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c49544-3780-4a1b-ba8a-afbb29b2a4e8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:47.532253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.532882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.533163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.533435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.534106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:47.535501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:47.556886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 533/1061] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:18:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:47.640302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:47.756425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9623e-8a26-46aa-b2ec-4021ad718913 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:47.758080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:47.798994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.799620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.799896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.800167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.812112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:47.818010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:47.818964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:47.819809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 529/1062] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Aug 24 03:18:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:47.825642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:47.943000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89642844-c9c4-461f-af2d-41a86dc1e0b8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:47.978186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:47.978786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:47.979063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:47.979334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:47.980260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:47.980559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:47.981961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:47.991685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 534/1063] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:18:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:47.997214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:48.104486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4346cf-eb45-4384-a575-bd2bc13fbe8f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:48.141506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:48.142134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.142408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.142674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.153500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:48.154946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:48.169865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:48.170929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:48.171852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:48.171852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:48.171852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:48.173091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 530/1064] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:48.177320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:48.190491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:48.283148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a0192-c16e-4837-a543-d00c56fafe86 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:48.317718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:48.318320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.318599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.318871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.319531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:48.319818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:48.321254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:48.338203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 535/1065] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:48.339556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:48.352964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:48.448900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fc56a-52a1-455c-9ce6-abc8edecfb2d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:48.484796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:48.485430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.485709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.485982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.495746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:48.498083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:48.514366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:48.515366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:48.521342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 531/1066] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:18:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:48.521940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:48.594782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:48.628265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2341-16f4-4fa5-889f-49d97e53f717 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:48.666127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:48.666743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.667020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.667293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.668044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:48.668350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:48.669986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:48.693426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 536/1067] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:18:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:48.694635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:48.766778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:48.800594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f5cdb-51fe-4f7a-b59f-6ea04bf397ab None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:48.835691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:48.836327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.836655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.836958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.848811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:48.850284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b8583c23-2e24-464c-8ecc-0520119b50bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:48.869585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:48.870585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:48.871484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:48.871484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:48.871484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:48.876866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 532/1068] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:48.877541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:48.952248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d661-9c97-454b-91e5-2f48374daae5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.952570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d661-9c97-454b-91e5-2f48374daae5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.952825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d661-9c97-454b-91e5-2f48374daae5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.954177 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 533/1069] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:48.959658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:48.960132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:48.960573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:48.979179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:48.980950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:48.994898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da9afd-c562-48b8-8fb9-6947231d92cc None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.023295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:49.033406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.034010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.034286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.034556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.035942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:49.036243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:49.037681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:49.049312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 537/1070] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:49.055435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.064847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 534/1071] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:49.157016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53b966-3bc6-446c-9d2f-eb085f1687e4 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.207551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.208513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.208807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.209082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.218612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:49.220114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:49.235129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:49.236036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:49.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:49.237414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 538/1072] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:49.242697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.298015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:49.339426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.348747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30590427-ae25-4bfd-80e7-3af18bf14406 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.383458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.384141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.384484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.384839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.385589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:49.385954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:49.387447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:49.391631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:49.405104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 535/1073] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:49.406085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.452136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc68f24-2911-4fbb-98e4-748af59aaf4d None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.491642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.492277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.492555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.492853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.505637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:49.510330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:49.511438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:49.512458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 539/1074] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:49.517368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.518546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73208-2fb9-4fb8-a50a-5b854315bf70 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.534626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:49.559820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b9c93-030c-4ae8-b94e-0dfa084c48fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.560479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b9c93-030c-4ae8-b94e-0dfa084c48fd None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.560779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b9c93-030c-4ae8-b94e-0dfa084c48fd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.561062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b9c93-030c-4ae8-b94e-0dfa084c48fd None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/b8afb31c267946b7851cb1c22b62b6cb/roles/4b669e9fe0fa42e59073f7443dadf4b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.563237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b9c93-030c-4ae8-b94e-0dfa084c48fd None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=b8afb31c267946b7851cb1c22b62b6cb, role_id=4b669e9fe0fa42e59073f7443dadf4b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:49.564685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b9c93-030c-4ae8-b94e-0dfa084c48fd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:49.578729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 536/1075] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:18:49 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/b8afb31c267946b7851cb1c22b62b6cb/roles/4b669e9fe0fa42e59073f7443dadf4b8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:18:49.579903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961405b0-47c3-4d6c-86d8-c3ea27eef035 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.580185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961405b0-47c3-4d6c-86d8-c3ea27eef035 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.580449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961405b0-47c3-4d6c-86d8-c3ea27eef035 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.581747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 537/1076] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:49.588122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.588404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.588698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.607674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:49.609143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:49.630353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990bb80-2207-4006-80f2-f7d45c57de37 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.640578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:49.675417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.676137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.676954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.676954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.677447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:49.677748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:49.679141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:49.694404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 540/1077] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:49.695634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.708976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 538/1078] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:49.801152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c89c1b-18eb-48ed-840c-4928f9ef5723 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.837088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:49.837774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:49.838044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:49.838313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] PATH_INFO: `/v3/users/369de73754ec4717aefa6d021243a6e7` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:49.839435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] RBAC: Authorizing `identity:get_user(user_id=369de73754ec4717aefa6d021243a6e7)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:49.840890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:49.842612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 541/1079] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/users/369de73754ec4717aefa6d021243a6e7 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:49.848800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.956635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa39d-7108-44ee-8d19-47526d44b592 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:49.958787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:49.970298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:49.999538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:50.000158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:50.000432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:50.000698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:50.001824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] RBAC: Authorizing `identity:get_project(project_id=af029992b398483096150f4f80112ff2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:50.003254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:50.005226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 539/1080] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/projects/af029992b398483096150f4f80112ff2 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:50.010951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:50.012600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:50.079090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676ea48-cf13-4300-9edf-ff10c7f57808 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:50.116683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:50.117561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:50.118007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:50.118423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:50.123365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b0fd-2196-4a3f-b425-a964c0ced3a0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:50.159410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:50.160767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:50.161317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:50.161317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2/users/369de73754ec4717aefa6d021243a6e7/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:50.163037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] RBAC: Authorizing `identity:create_grant(project_id=af029992b398483096150f4f80112ff2, user_id=369de73754ec4717aefa6d021243a6e7, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:50.164454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:50.178762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 540/1081] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:18:50 2022] PUT /identity/v3/projects/af029992b398483096150f4f80112ff2/users/369de73754ec4717aefa6d021243a6e7/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:18:50.209465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:50.211150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:50.256372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:50.326208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 542/1082] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:18:50.332749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:50.372321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:50.446628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84527ec8-7212-4672-ba78-a43ce4be40ca None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:50.482693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:50.483327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:50.483606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:50.483876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:50.494433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:50.495899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e467effa-6b4b-4568-aff1-bc2e72210ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:50.510976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:50.511945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:50.512854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:50.512854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:50.512854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:50.512854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 541/1083] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:50.518265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:50.627507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517726a-e159-4762-a29f-0241bf364fff None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:50.665543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:50.666410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:50.666410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:50.666682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:50.667819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:50.668133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:50.669577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae9f7-c8c5-41ac-b760-6207cb5ea96f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:50.678755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:50.693332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 543/1084] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:51.511890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87a94-c2bb-4487-9abb-4cada5bbbc5e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:51.512414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87a94-c2bb-4487-9abb-4cada5bbbc5e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:51.512414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87a94-c2bb-4487-9abb-4cada5bbbc5e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:51.513794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 542/1085] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:51.518916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:51.519336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:51.519595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:51.537651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:51.538836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:51.606184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 544/1086] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:51.875245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf87a94-c2bb-4487-9abb-4cada5bbbc5e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:51.959373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:51.979345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf87a94-c2bb-4487-9abb-4cada5bbbc5e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.014253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44930a3a-8c7f-4acd-ac2b-4705aa04fc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.014871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44930a3a-8c7f-4acd-ac2b-4705aa04fc9b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.015148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44930a3a-8c7f-4acd-ac2b-4705aa04fc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.015417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44930a3a-8c7f-4acd-ac2b-4705aa04fc9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.025783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44930a3a-8c7f-4acd-ac2b-4705aa04fc9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-44930a3a-8c7f-4acd-ac2b-4705aa04fc9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:52.030418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:52.031349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:52.032265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 543/1087] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Aug 24 03:18:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:18:52.038318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.124486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641d6d6-4a12-49af-9308-9bcc950e88f3 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.124783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641d6d6-4a12-49af-9308-9bcc950e88f3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.125069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641d6d6-4a12-49af-9308-9bcc950e88f3 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.126465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 544/1088] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:52.132093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.132378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.132669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.142347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678379d3-c83e-42e3-b86d-5fc50c37e9de None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.151925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:52.153445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:52.177458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:52.186959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.187566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.187851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.188120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.188814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:52.189123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.190496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.200098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 545/1089] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:52.205764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.228647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 545/1090] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:18:52.305232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b15220-f075-463d-a75e-a6780c3f5330 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.339742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.340360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.340662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.340959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.351403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.352861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:52.367154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:52.368116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:52.369044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:52.369044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:52.369044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:52.369044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 546/1091] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:52.374283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.444896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:52.474644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2a972-31f2-469e-8523-403f8d4c91cb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.493790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.518320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.518928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.519206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.519486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.520176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:52.520474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.521918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.534495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 546/1092] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:52.539721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.542416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:52.600537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2eaff-cb9e-4354-8fdf-ecd75c616bc2 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.635390 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.636012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.636291 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.636561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.642277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20809d0a-934d-4cca-bd64-18481b6b92c1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.647006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:18:52.651614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:52.652552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:18:52.653463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 547/1093] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:52.658728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.671697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:52.678371 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.679100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.679386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.679658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.687192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.688635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:18:52.700179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:52.701318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:52.702271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 547/1094] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:52.708418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.767581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e588b78-4152-4f9d-9c58-856084bf6693 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.769127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:52.802786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.803386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.803668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.803932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.804594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:52.804924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.806298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.809318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c2bf-8ea2-41e3-8008-95ad5f9cd9f3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.814718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 548/1095] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:52.821074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.844700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.845331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.845616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.845890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.846610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:52.846916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.848484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.866281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 548/1096] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:52.871943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.923407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:52.925789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e7ed5-3168-4c6c-80cd-ef92c95a5cec None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:52.960889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:52.961465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:52.961742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:52.962018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] PATH_INFO: `/v3/users/369de73754ec4717aefa6d021243a6e7` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:52.963187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] RBAC: Authorizing `identity:get_user(user_id=369de73754ec4717aefa6d021243a6e7)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:52.964620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:52.966391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 549/1097] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/users/369de73754ec4717aefa6d021243a6e7 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:52.971932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:52.974758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea95d7-2691-4aa9-b67f-a9ec7fb1ee7f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.002688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.008249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.008876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.009155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.009421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.073609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c15ee-e4a1-4fad-84d9-c2ac22dfc3a0 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.077150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.078628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.110117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.110799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.111082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.111359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.129970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.179570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.181038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.183362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 549/1098] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:18:53.189167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.212466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.283572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 550/1099] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:18:53.289742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bc549-0850-47e9-a5e2-e19ba744b236 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.290106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.323155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.323766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.324035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.324301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.335541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.337024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:18:53.351613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:53.352638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:18:53.353588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:18:53.353588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:18:53.353588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:53.353588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 550/1100] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:53.358516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.361126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.393104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26317bb-f818-4d27-9053-e66c0c16bb04 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.422401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.427641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.428262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.428531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.428829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.429941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] RBAC: Authorizing `identity:get_project(project_id=af029992b398483096150f4f80112ff2)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.431376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.433068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 551/1101] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/projects/af029992b398483096150f4f80112ff2 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:53.438971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.462977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb181a5-4f22-4dca-af9f-bceaa5d007e0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.497209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.497887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.498168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.498438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.499068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:53.499410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.500722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.513411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 551/1102] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:53.518961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.557221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f42d15-0788-4400-bb0e-24c8530b76e8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.588496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.595315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.596012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.596321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.596596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.597372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=af029992b398483096150f4f80112ff2, user.id=369de73754ec4717aefa6d021243a6e7) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:53.597679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.599519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.605668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 552/1103] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/role_assignments?user.id=369de73754ec4717aefa6d021243a6e7&scope.project.id=af029992b398483096150f4f80112ff2&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:18:53.632588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62c00a-765e-42d8-8626-79af2febe634 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.671540 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.672182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.672459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.672730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.680816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.682315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:18:53.694953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:18:53.696081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:18:53.697041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 552/1104] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:18:53.702203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.769456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:53.810844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcaa21-3551-4bf2-837b-e7be985cc82e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:53.844089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:53.844677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:53.844987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:53.845256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:53.845934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:53.846233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:53.847648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d9c02-e77f-4af2-8689-bef0b3bf9f76 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:53.860570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 553/1105] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:53.867268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:53.981666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82d04a-3f46-4584-a8e2-60f971c8ed0e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:54.004009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:54.016378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:54.016992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:54.017263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:54.017527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:54.018238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] RBAC: Adding query filter params (role.id=4b669e9fe0fa42e59073f7443dadf4b8, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=b8afb31c267946b7851cb1c22b62b6cb) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:54.018537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:54.020170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132a5ad-9f92-4a3a-8897-1a52c9934396 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:54.026033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 553/1106] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:53 2022] GET /identity/v3/role_assignments?user.id=b8afb31c267946b7851cb1c22b62b6cb&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=4b669e9fe0fa42e59073f7443dadf4b8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:55.559848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d351-87b2-4858-8a90-277d2eb54853 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:55.560453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d351-87b2-4858-8a90-277d2eb54853 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:55.560453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d351-87b2-4858-8a90-277d2eb54853 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:55.561778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 554/1107] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:55.568008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:55.568454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:55.568454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:55.586651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:55.587844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:55.653752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 554/1108] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:55.925146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292d351-87b2-4858-8a90-277d2eb54853 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:56.026014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292d351-87b2-4858-8a90-277d2eb54853 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:56.050046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:56.061095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ed019-14a2-432c-b199-5481beaea7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:56.061705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ed019-14a2-432c-b199-5481beaea7cf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:56.061981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ed019-14a2-432c-b199-5481beaea7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:56.062255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ed019-14a2-432c-b199-5481beaea7cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:56.063361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ed019-14a2-432c-b199-5481beaea7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:56.064810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ed019-14a2-432c-b199-5481beaea7cf None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:56.066371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 555/1109] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:18:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:56.072134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:56.169635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c398a0-bf44-46e1-ad6a-faea9a17274e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:56.185895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:56.204125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36406a83-0f67-4b02-a0a3-8bf766f23eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:56.204718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36406a83-0f67-4b02-a0a3-8bf766f23eb0 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:56.205022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36406a83-0f67-4b02-a0a3-8bf766f23eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:56.205292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36406a83-0f67-4b02-a0a3-8bf766f23eb0 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:56.205923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36406a83-0f67-4b02-a0a3-8bf766f23eb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:56.207349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36406a83-0f67-4b02-a0a3-8bf766f23eb0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:56.228190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 555/1110] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:18:56 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:18:58.186835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046324f7-e543-403e-bbf9-9f00ae726cb8 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:58.187340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046324f7-e543-403e-bbf9-9f00ae726cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:58.187340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046324f7-e543-403e-bbf9-9f00ae726cb8 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:58.188638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 556/1111] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:18:58.193977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:58.194364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:58.194516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:58.212588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:18:58.213920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:18:58.279384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 556/1112] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:18:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:18:58.549969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046324f7-e543-403e-bbf9-9f00ae726cb8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:58.648191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046324f7-e543-403e-bbf9-9f00ae726cb8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:58.681436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:58.682033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:58.682306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:58.682574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] PATH_INFO: `/v3/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:58.683926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] RBAC: Authorizing `identity:get_role(role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:58.685302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:58.686582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 557/1113] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:18:58 2022] GET /identity/v3/roles/7eef59c95b8945768aab3374bc2721f1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:58.692809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:58.788747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c269-d119-453f-8b34-e40b11531430 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:58.790258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:58.823823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:58.824373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:58.824650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:58.824949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] PATH_INFO: `/v3/users/cb8c1283d411482fb52948289e7b2fb4` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:58.838231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] RBAC: Authorizing `identity:get_user(user_id=cb8c1283d411482fb52948289e7b2fb4)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:58.839675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:58.841296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 557/1114] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:18:58 2022] GET /identity/v3/users/cb8c1283d411482fb52948289e7b2fb4 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:58.847065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:58.906627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:58.948674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31908935-43bf-4a3b-b2bc-798c59590650 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:58.981937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:58.982521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:58.982798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:58.983068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:59.048623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:59.050085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:59.105484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:18:59.147510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 558/1115] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:18:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:18:59.153553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:59.249848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dd83e-df3e-4c63-a2e7-72defe25ebdb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:59.284066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efda934-afb9-414c-b22a-51df16bf0865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:59.284668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efda934-afb9-414c-b22a-51df16bf0865 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:59.285095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efda934-afb9-414c-b22a-51df16bf0865 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:59.285362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efda934-afb9-414c-b22a-51df16bf0865 None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:59.287001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efda934-afb9-414c-b22a-51df16bf0865 None admin] RBAC: Authorizing `identity:get_project(project_id=af029992b398483096150f4f80112ff2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:59.288408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efda934-afb9-414c-b22a-51df16bf0865 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:59.290040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 558/1116] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:18:59 2022] GET /identity/v3/projects/af029992b398483096150f4f80112ff2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:18:59.296164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:18:59.393222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7b34-e765-43ef-98f7-8503b51e6601 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:18:59.426338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:18:59.426927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:18:59.427199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:18:59.427469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:18:59.428190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] RBAC: Adding query filter params (role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=af029992b398483096150f4f80112ff2, user.id=cb8c1283d411482fb52948289e7b2fb4) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:18:59.428496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:18:59.429943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6dc163-0d55-4f8b-9e6b-f2ab7d6c96d6 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:18:59.435163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 559/1117] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:18:59 2022] GET /identity/v3/role_assignments?user.id=cb8c1283d411482fb52948289e7b2fb4&scope.project.id=af029992b398483096150f4f80112ff2&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:01.309994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905904d-93de-49a8-958d-30cb421dc022 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:01.310506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905904d-93de-49a8-958d-30cb421dc022 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:01.310506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905904d-93de-49a8-958d-30cb421dc022 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:01.311823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 559/1118] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:01.316915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:01.317317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:01.317468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:01.336070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:01.337302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:01.355082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:01.403624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 560/1119] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:01.674948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905904d-93de-49a8-958d-30cb421dc022 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:01.703159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:01.775805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905904d-93de-49a8-958d-30cb421dc022 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:01.810103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:01.810710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:01.810987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:01.811260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] PATH_INFO: `/v3/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:01.812644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] RBAC: Authorizing `identity:get_role(role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:01.814038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:01.815347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 560/1120] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:01 2022] GET /identity/v3/roles/7eef59c95b8945768aab3374bc2721f1 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:01.821552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:01.919989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb37d-afaf-426d-ac86-52acf6f7bf27 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:01.954169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:01.954708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:01.954989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:01.955258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] PATH_INFO: `/v3/users/cb8c1283d411482fb52948289e7b2fb4` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:01.956376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=cb8c1283d411482fb52948289e7b2fb4)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:01.957840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:01.959521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 561/1121] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:01 2022] GET /identity/v3/users/cb8c1283d411482fb52948289e7b2fb4 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:01.965462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:02.058133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:02.064411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e72682-6fa2-4ce2-8031-e624ff3c4b02 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:02.098130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31badd1d-96a0-4e7d-a2be-19bf64243593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:02.098768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31badd1d-96a0-4e7d-a2be-19bf64243593 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:02.099646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31badd1d-96a0-4e7d-a2be-19bf64243593 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:02.099940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31badd1d-96a0-4e7d-a2be-19bf64243593 None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:02.101139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31badd1d-96a0-4e7d-a2be-19bf64243593 None admin] RBAC: Authorizing `identity:get_project(project_id=af029992b398483096150f4f80112ff2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:02.102578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31badd1d-96a0-4e7d-a2be-19bf64243593 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:02.104195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 561/1122] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:01 2022] GET /identity/v3/projects/af029992b398483096150f4f80112ff2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:02.110435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:02.134377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:02.206749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50269e4f-2483-4ea6-979f-215a6d618eb7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:02.240409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d1da3-9850-4713-ad82-6bffb90f5e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:02.241062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d1da3-9850-4713-ad82-6bffb90f5e3e None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:02.241336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d1da3-9850-4713-ad82-6bffb90f5e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:02.241603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d1da3-9850-4713-ad82-6bffb90f5e3e None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2/users/cb8c1283d411482fb52948289e7b2fb4/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:02.243690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d1da3-9850-4713-ad82-6bffb90f5e3e None admin] RBAC: Authorizing `identity:create_grant(project_id=af029992b398483096150f4f80112ff2, user_id=cb8c1283d411482fb52948289e7b2fb4, role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:02.245128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d1da3-9850-4713-ad82-6bffb90f5e3e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:02.257241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 562/1123] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:19:02 2022] PUT /identity/v3/projects/af029992b398483096150f4f80112ff2/users/cb8c1283d411482fb52948289e7b2fb4/roles/7eef59c95b8945768aab3374bc2721f1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:19:04.122999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4d90-c951-4655-bec4-b0f61fe76d48 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:04.123510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4d90-c951-4655-bec4-b0f61fe76d48 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:04.123631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4d90-c951-4655-bec4-b0f61fe76d48 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:04.124919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 562/1124] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:04.129992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:04.130370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:04.130579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:04.148129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:04.149333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:04.214570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 563/1125] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:04.487265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4d90-c951-4655-bec4-b0f61fe76d48 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:04.588942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4d90-c951-4655-bec4-b0f61fe76d48 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:04.623079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:04.623700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:04.623979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:04.624254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] PATH_INFO: `/v3/roles/7eef59c95b8945768aab3374bc2721f1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:04.625701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] RBAC: Authorizing `identity:get_role(role_id=7eef59c95b8945768aab3374bc2721f1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:04.627071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:04.628383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 563/1126] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:04 2022] GET /identity/v3/roles/7eef59c95b8945768aab3374bc2721f1 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:19:04.634285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:04.648206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:04.736778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f75c0b-fad9-4f91-aad4-0848166405cb None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:04.769831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:04.770375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:04.770647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:04.770913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] PATH_INFO: `/v3/users/cb8c1283d411482fb52948289e7b2fb4` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:04.772560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] RBAC: Authorizing `identity:get_user(user_id=cb8c1283d411482fb52948289e7b2fb4)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:04.774013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:04.775566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 564/1127] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:04 2022] GET /identity/v3/users/cb8c1283d411482fb52948289e7b2fb4 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:04.781463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:04.813568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:04.883181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4881881-5b3b-4605-bdbf-6e36d8fb7358 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:04.917455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:04.918053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:04.918325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:04.918593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:04.985205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:04.986649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:05.015968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:05.086166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 564/1128] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:19:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:19:05.092152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:05.172257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:05.191345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b6512-a57b-43b6-9f54-d15db2fc1176 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:05.225343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf74f-4714-4497-9e84-a6ee1a8cb696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:05.225962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf74f-4714-4497-9e84-a6ee1a8cb696 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:05.226241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf74f-4714-4497-9e84-a6ee1a8cb696 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:05.226515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf74f-4714-4497-9e84-a6ee1a8cb696 None admin] PATH_INFO: `/v3/projects/af029992b398483096150f4f80112ff2` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:05.227635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf74f-4714-4497-9e84-a6ee1a8cb696 None admin] RBAC: Authorizing `identity:get_project(project_id=af029992b398483096150f4f80112ff2)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:05.229090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf74f-4714-4497-9e84-a6ee1a8cb696 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:05.230702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 565/1129] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:05 2022] GET /identity/v3/projects/af029992b398483096150f4f80112ff2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:05.237094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:05.335350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab03095-77b3-4a8f-b7d4-d3767a0fc603 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:05.364372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:05.369914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:05.370502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:05.370773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:05.371040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:05.371749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] RBAC: Adding query filter params (role.id=7eef59c95b8945768aab3374bc2721f1, scope.project.id=af029992b398483096150f4f80112ff2, user.id=cb8c1283d411482fb52948289e7b2fb4) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:05.372049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:05.373507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f72971-6206-4256-bdc1-7183cdeaaa2c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:05.379133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 565/1130] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:19:05 2022] GET /identity/v3/role_assignments?user.id=cb8c1283d411482fb52948289e7b2fb4&scope.project.id=af029992b398483096150f4f80112ff2&role.id=7eef59c95b8945768aab3374bc2721f1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:07.284857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430f09-e3b8-4f87-a904-00baf95e7bf5 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:07.285372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430f09-e3b8-4f87-a904-00baf95e7bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:07.285372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430f09-e3b8-4f87-a904-00baf95e7bf5 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:07.286674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 566/1131] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:07.292249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:07.292658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:07.292800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:07.310750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:07.312073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:07.382119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 566/1132] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:07.653623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430f09-e3b8-4f87-a904-00baf95e7bf5 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:07.756598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430f09-e3b8-4f87-a904-00baf95e7bf5 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:07.790263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abef425-0fb3-4d4a-9a02-1b631e52e9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:07.790856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abef425-0fb3-4d4a-9a02-1b631e52e9fc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:07.791127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abef425-0fb3-4d4a-9a02-1b631e52e9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:07.791391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abef425-0fb3-4d4a-9a02-1b631e52e9fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:07.792478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abef425-0fb3-4d4a-9a02-1b631e52e9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:07.793925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abef425-0fb3-4d4a-9a02-1b631e52e9fc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:07.795459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 567/1133] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:19:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:07.801302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:07.900283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4057e9-791e-4519-a436-19840ca88f9e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:07.925427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:07.934993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23c096-7957-46ad-8606-6258bf700c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:07.935579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c096-7957-46ad-8606-6258bf700c0c None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:07.935851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c096-7957-46ad-8606-6258bf700c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:07.936116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c096-7957-46ad-8606-6258bf700c0c None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:07.936753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23c096-7957-46ad-8606-6258bf700c0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:07.938194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23c096-7957-46ad-8606-6258bf700c0c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:07.965678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 567/1134] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:19:07 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:19:09.921808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c75ff-cdce-4be8-ab99-b42826ca9fde None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:09.922247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c75ff-cdce-4be8-ab99-b42826ca9fde None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:09.922354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c75ff-cdce-4be8-ab99-b42826ca9fde None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:09.923694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 568/1135] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:09.930344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:09.930777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:09.930881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:09.949190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:09.950371 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:10.017186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 568/1136] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:10.286369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c75ff-cdce-4be8-ab99-b42826ca9fde None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:10.304886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:10.388035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c75ff-cdce-4be8-ab99-b42826ca9fde None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:10.421341 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0505f-a501-458a-a319-b72fe85e80a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:10.421948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0505f-a501-458a-a319-b72fe85e80a2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:10.422224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0505f-a501-458a-a319-b72fe85e80a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:10.422494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0505f-a501-458a-a319-b72fe85e80a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:10.423590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0505f-a501-458a-a319-b72fe85e80a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:10.425052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0505f-a501-458a-a319-b72fe85e80a2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:10.426608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 569/1137] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:19:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:10.432890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:10.531138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb55dc-e615-4c8b-86cc-b665bea21479 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:10.541005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:10.567299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b71fac-1f6a-4b7b-a569-68eeb6a612f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:10.567894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b71fac-1f6a-4b7b-a569-68eeb6a612f1 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:10.568171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b71fac-1f6a-4b7b-a569-68eeb6a612f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:10.568443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b71fac-1f6a-4b7b-a569-68eeb6a612f1 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:10.569920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b71fac-1f6a-4b7b-a569-68eeb6a612f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:10.571588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b71fac-1f6a-4b7b-a569-68eeb6a612f1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:10.593042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 569/1138] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:19:10 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:19:12.518203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd247086-68c4-4180-b199-0e4eb6cfe532 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:12.518728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd247086-68c4-4180-b199-0e4eb6cfe532 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:12.518728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd247086-68c4-4180-b199-0e4eb6cfe532 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:12.520077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 570/1139] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:12.525531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:12.525957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:12.526081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:12.543748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:12.544969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:12.612444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 570/1140] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:12.884317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd247086-68c4-4180-b199-0e4eb6cfe532 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:12.969420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:12.987072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd247086-68c4-4180-b199-0e4eb6cfe532 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:13.020450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:13.021092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:13.021370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:13.021641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:13.031657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:19:13.036228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:13.037990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:19:13.039219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 571/1141] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:13.044036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:13.134750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:13.142469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b59bb-422e-4290-94e8-006511ee6d0d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:13.175708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:13.176294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:13.176571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:13.176878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:13.177522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:13.177818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:13.179896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:13.187733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 571/1142] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:19:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:13.193458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:13.292842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944654e-592b-4de4-b426-d158586f2399 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:13.326836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:13.327438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:13.327710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:13.327976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] PATH_INFO: `/v3/users/2a184734fed9449bbc8f3b900d361eda` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:13.340695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] RBAC: Authorizing `identity:get_user(user_id=2a184734fed9449bbc8f3b900d361eda)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:13.342135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:13.343690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 572/1143] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:13 2022] GET /identity/v3/users/2a184734fed9449bbc8f3b900d361eda => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:13.349625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:13.450064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c2430-de24-4d6a-a2d9-1958352cbac5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:13.488683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:13.489315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:13.489590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:13.489859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:13.516649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:13.559581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:13.561065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:13.656872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 572/1144] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:19:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:19:13.663388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:13.666245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:13.763788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03beacb5-110d-4b43-b897-05edcdbd9092 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:13.807433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83b02-0138-4c76-a461-e54bd268f394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:13.808076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83b02-0138-4c76-a461-e54bd268f394 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:13.808352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83b02-0138-4c76-a461-e54bd268f394 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:13.808627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83b02-0138-4c76-a461-e54bd268f394 None admin] PATH_INFO: `/v3/projects/45a27892520f4db39a6608820b0a9a8a` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:13.809785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83b02-0138-4c76-a461-e54bd268f394 None admin] RBAC: Authorizing `identity:get_project(project_id=45a27892520f4db39a6608820b0a9a8a)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:13.811226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83b02-0138-4c76-a461-e54bd268f394 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:13.812906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 573/1145] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:13 2022] GET /identity/v3/projects/45a27892520f4db39a6608820b0a9a8a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:13.819110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:13.874469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:13.918365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba12e4-94d8-42fd-ae7b-21cec5c64455 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:13.951533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:13.952116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:13.952383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:13.952646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:13.953382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=45a27892520f4db39a6608820b0a9a8a, user.id=2a184734fed9449bbc8f3b900d361eda) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:13.953682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:13.955098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb13f3-49f7-4094-bd91-742c72b32603 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:13.961760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 573/1146] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:19:13 2022] GET /identity/v3/role_assignments?user.id=2a184734fed9449bbc8f3b900d361eda&scope.project.id=45a27892520f4db39a6608820b0a9a8a&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:15.870125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee70a1-93a0-4d14-8aaf-c68026bc22b0 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:15.870679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee70a1-93a0-4d14-8aaf-c68026bc22b0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:15.870679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee70a1-93a0-4d14-8aaf-c68026bc22b0 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:15.872008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 574/1147] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:15.877210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:15.877606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:15.877757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:15.897286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:15.898566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:15.965011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 574/1148] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:16.234468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee70a1-93a0-4d14-8aaf-c68026bc22b0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:16.304569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:16.336976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee70a1-93a0-4d14-8aaf-c68026bc22b0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:16.370374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:16.370980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:16.371252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:16.371514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:16.381883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:19:16.386399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:16.387531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:19:16.388584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 575/1149] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:16.393708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:16.443357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:16.492673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bab01-2b50-4041-9313-dc8caa2801fc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:16.525307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:16.525887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:16.526156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:16.526420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:16.527064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:16.527385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:16.528739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:16.536478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 575/1150] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:16.543014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:16.643966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a321e-15eb-47c1-b22b-6f7574b5c057 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:16.672523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:16.679678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:16.680282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:16.680559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:16.680863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] PATH_INFO: `/v3/users/2a184734fed9449bbc8f3b900d361eda` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:16.681976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] RBAC: Authorizing `identity:get_user(user_id=2a184734fed9449bbc8f3b900d361eda)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:16.683401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:16.684988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 576/1151] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:16 2022] GET /identity/v3/users/2a184734fed9449bbc8f3b900d361eda => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:16.690887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:16.788841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc3bd-b170-4a4f-b647-131c0a05d1ff None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:16.801161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:16.823026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefcc43-13cc-445c-9736-48c3a0674105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:16.823704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefcc43-13cc-445c-9736-48c3a0674105 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:16.823977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefcc43-13cc-445c-9736-48c3a0674105 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:16.824249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefcc43-13cc-445c-9736-48c3a0674105 None admin] PATH_INFO: `/v3/projects/45a27892520f4db39a6608820b0a9a8a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:16.825363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefcc43-13cc-445c-9736-48c3a0674105 None admin] RBAC: Authorizing `identity:get_project(project_id=45a27892520f4db39a6608820b0a9a8a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:16.826798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefcc43-13cc-445c-9736-48c3a0674105 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:16.828395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 576/1152] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:16 2022] GET /identity/v3/projects/45a27892520f4db39a6608820b0a9a8a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:16.834711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:16.934334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68aac26-4489-4f87-8d76-072f21025f6c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:16.967713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635037cb-221f-41b6-b532-b564bcf4ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:16.968349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635037cb-221f-41b6-b532-b564bcf4ea83 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:16.968629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635037cb-221f-41b6-b532-b564bcf4ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:16.968937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635037cb-221f-41b6-b532-b564bcf4ea83 None admin] PATH_INFO: `/v3/projects/45a27892520f4db39a6608820b0a9a8a/users/2a184734fed9449bbc8f3b900d361eda/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:16.970840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635037cb-221f-41b6-b532-b564bcf4ea83 None admin] RBAC: Authorizing `identity:create_grant(project_id=45a27892520f4db39a6608820b0a9a8a, user_id=2a184734fed9449bbc8f3b900d361eda, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:16.972271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635037cb-221f-41b6-b532-b564bcf4ea83 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:16.987552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 577/1153] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:19:16 2022] PUT /identity/v3/projects/45a27892520f4db39a6608820b0a9a8a/users/2a184734fed9449bbc8f3b900d361eda/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:19:18.918474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7fde-e594-4fd3-9f7d-5b0c2a00f96b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:18.918915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7fde-e594-4fd3-9f7d-5b0c2a00f96b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:18.919047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7fde-e594-4fd3-9f7d-5b0c2a00f96b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:18.920393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 577/1154] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:18.925776 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:18.926171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:18.926316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:18.944326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:18.945519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:18.996932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:19.013406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 578/1155] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:19.290475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7fde-e594-4fd3-9f7d-5b0c2a00f96b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:19.391435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7fde-e594-4fd3-9f7d-5b0c2a00f96b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:19.424919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:19.425524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:19.425798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:19.426068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:19.438022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:19:19.443146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:19.444889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:19:19.446608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 578/1156] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:19:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:19.451710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:19.572898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560b1a2-3353-4452-aaa3-714e595d3b4f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:19.606589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:19.607179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:19.607452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:19.607719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:19.608372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:19.608670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:19.610062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:19.618808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 579/1157] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:19.624636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:19.665695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:19.723191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a6a0a-3d01-42cd-852a-503eb1227364 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:19.756862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:19.757471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:19.757747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:19.758019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] PATH_INFO: `/v3/users/2a184734fed9449bbc8f3b900d361eda` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:19.759129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] RBAC: Authorizing `identity:get_user(user_id=2a184734fed9449bbc8f3b900d361eda)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:19.760568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:19.762173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 579/1158] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:19 2022] GET /identity/v3/users/2a184734fed9449bbc8f3b900d361eda => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:19.768185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:19.821323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:19.873574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4841bce-93fe-4feb-ac39-1f64163c6931 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:19.911324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:19.911978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:19.912256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:19.912528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:19.979503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:19.980976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:20.028482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:20.078645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 580/1159] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:19:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:19:20.085342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:20.189337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42572874-39d2-45cf-a389-11e81da5d0e5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:20.198333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:20.223392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b061faa-0197-46d1-8ee1-7c7c9512bb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:20.224538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b061faa-0197-46d1-8ee1-7c7c9512bb53 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:20.224844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b061faa-0197-46d1-8ee1-7c7c9512bb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:20.225120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b061faa-0197-46d1-8ee1-7c7c9512bb53 None admin] PATH_INFO: `/v3/projects/45a27892520f4db39a6608820b0a9a8a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:20.226251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b061faa-0197-46d1-8ee1-7c7c9512bb53 None admin] RBAC: Authorizing `identity:get_project(project_id=45a27892520f4db39a6608820b0a9a8a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:20.227684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b061faa-0197-46d1-8ee1-7c7c9512bb53 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:20.229365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 580/1160] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:20 2022] GET /identity/v3/projects/45a27892520f4db39a6608820b0a9a8a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:20.235737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:20.331987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca9a8f-e4c1-4eb7-8e24-6c063a0843fc None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:20.366669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:20.367242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:20.367512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:20.367776 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:20.368481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=45a27892520f4db39a6608820b0a9a8a, user.id=2a184734fed9449bbc8f3b900d361eda) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:20.368793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:20.370212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef44c0d-9a55-4144-ab6e-2e9c6296b7c1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:20.375723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 581/1161] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:19:20 2022] GET /identity/v3/role_assignments?user.id=2a184734fed9449bbc8f3b900d361eda&scope.project.id=45a27892520f4db39a6608820b0a9a8a&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:22.353334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ae02-401a-4330-98ab-e820ab2acc21 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:22.353832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ae02-401a-4330-98ab-e820ab2acc21 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:22.353832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ae02-401a-4330-98ab-e820ab2acc21 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:22.355116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 581/1162] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:22.360477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:22.360894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:22.361049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:22.378791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:22.380108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:22.389248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:22.446684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 582/1163] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:22.729747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ae02-401a-4330-98ab-e820ab2acc21 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:22.834011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ae02-401a-4330-98ab-e820ab2acc21 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:22.867278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:22.867857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:22.868130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:22.868401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:22.914248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:22.937449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:22.938899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:23.037925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 582/1164] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:19:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:19:23.043805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:23.142360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6db5d-742a-4c82-86c2-8161dcb10619 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:23.176053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:23.176817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:23.177097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:23.177363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:23.183771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:23.185222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-742a26f8-b29a-40e8-a6fc-2180d942fd3d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:23.196188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:23.197462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:19:23.198332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 583/1165] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:19:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:23.203158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:23.277568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:23.301071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e497a2-7309-476c-894e-9ebdc099bcac None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:23.334541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:23.335124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:23.335396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:23.335664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:23.336286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:23.336581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:23.337973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be204c7-4a57-44f7-865f-4101099f5ec1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:23.344880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 583/1166] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Aug 24 03:19:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:25.258639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa34dd2-7991-4d90-9417-894802a89188 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:25.259152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa34dd2-7991-4d90-9417-894802a89188 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:25.259152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa34dd2-7991-4d90-9417-894802a89188 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:25.260423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 584/1167] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:25.265752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2844c-2fae-443a-98a7-7506ea7d5e4b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:25.266170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2844c-2fae-443a-98a7-7506ea7d5e4b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:25.266288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2844c-2fae-443a-98a7-7506ea7d5e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:25.284179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-75a2844c-2fae-443a-98a7-7506ea7d5e4b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:25.285983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-75a2844c-2fae-443a-98a7-7506ea7d5e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:25.350283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 584/1168] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:25.619230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa34dd2-7991-4d90-9417-894802a89188 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:25.623115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:25.719541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa34dd2-7991-4d90-9417-894802a89188 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:25.753284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d36a8-6038-40bd-8c79-f3cd5f574b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:25.753970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d36a8-6038-40bd-8c79-f3cd5f574b59 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:25.754251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d36a8-6038-40bd-8c79-f3cd5f574b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:25.754523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d36a8-6038-40bd-8c79-f3cd5f574b59 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:25.755200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d36a8-6038-40bd-8c79-f3cd5f574b59 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:25.756560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d36a8-6038-40bd-8c79-f3cd5f574b59 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:25.772831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 585/1169] 10.222.0.81 () {60 vars in 1138 bytes} [Wed Aug 24 03:19:25 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:19:27.719078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb59f-8010-40ab-9067-db4411e9ef69 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:27.719638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb59f-8010-40ab-9067-db4411e9ef69 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:27.719638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb59f-8010-40ab-9067-db4411e9ef69 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:27.721139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 585/1170] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:27.726563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:27.726997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:27.727101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:27.745543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:27.746711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:27.804353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:27.813588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 586/1171] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:28.092917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb59f-8010-40ab-9067-db4411e9ef69 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:28.164652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:28.195968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb59f-8010-40ab-9067-db4411e9ef69 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:28.229301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c300ff6-eaac-42ad-b84b-e5e9dbe2ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:28.230168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c300ff6-eaac-42ad-b84b-e5e9dbe2ad91 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:28.230438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c300ff6-eaac-42ad-b84b-e5e9dbe2ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:28.230704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c300ff6-eaac-42ad-b84b-e5e9dbe2ad91 None admin] PATH_INFO: `/v3/domains/5a9fa6c10d534d298b472728bbb85eee` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:28.240657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c300ff6-eaac-42ad-b84b-e5e9dbe2ad91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5a9fa6c10d534d298b472728bbb85eee)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:28.242274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c300ff6-eaac-42ad-b84b-e5e9dbe2ad91 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:28.243830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 586/1172] 10.222.0.81 () {56 vars in 1183 bytes} [Wed Aug 24 03:19:28 2022] GET /identity/v3/domains/5a9fa6c10d534d298b472728bbb85eee => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:28.249713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:28.347336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace656-ea69-4973-9e94-9cd4415dda03 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:28.379812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8bc92-37ce-4e2e-b218-c8f96b509608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:28.380391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8bc92-37ce-4e2e-b218-c8f96b509608 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:28.380665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8bc92-37ce-4e2e-b218-c8f96b509608 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:28.380961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8bc92-37ce-4e2e-b218-c8f96b509608 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:28.381590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8bc92-37ce-4e2e-b218-c8f96b509608 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:28.383007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8bc92-37ce-4e2e-b218-c8f96b509608 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:28.409781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 587/1173] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:19:28 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:19:30.324217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb3ff0-be21-4474-8345-2f90ea8fbbf3 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:30.324730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb3ff0-be21-4474-8345-2f90ea8fbbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:30.324730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb3ff0-be21-4474-8345-2f90ea8fbbf3 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:30.326087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 587/1174] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:30.332204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:30.332704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:30.332838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:30.350544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:30.351713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:30.392698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:30.418191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 588/1175] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:30.689204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb3ff0-be21-4474-8345-2f90ea8fbbf3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:30.792783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb3ff0-be21-4474-8345-2f90ea8fbbf3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:30.821987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:30.828191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6b820-78d5-46fc-acee-69cf8520335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:30.828829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6b820-78d5-46fc-acee-69cf8520335b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:30.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6b820-78d5-46fc-acee-69cf8520335b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:30.829370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6b820-78d5-46fc-acee-69cf8520335b None admin] PATH_INFO: `/v3/domains/5a9fa6c10d534d298b472728bbb85eee` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:30.830477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6b820-78d5-46fc-acee-69cf8520335b None admin] RBAC: Authorizing `identity:get_domain(domain_id=5a9fa6c10d534d298b472728bbb85eee)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:30.832012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6b820-78d5-46fc-acee-69cf8520335b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:30.833605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 588/1176] 10.222.0.81 () {56 vars in 1183 bytes} [Wed Aug 24 03:19:30 2022] GET /identity/v3/domains/5a9fa6c10d534d298b472728bbb85eee => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:30.839298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:30.936168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b566e-c2d5-4f20-813d-a762c73d62f5 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:30.969469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b073d-35b4-4625-984c-cf3aa7b0f85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:30.970000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b073d-35b4-4625-984c-cf3aa7b0f85d None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:30.970276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b073d-35b4-4625-984c-cf3aa7b0f85d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:30.970544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b073d-35b4-4625-984c-cf3aa7b0f85d None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:30.971910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b073d-35b4-4625-984c-cf3aa7b0f85d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:30.973454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b073d-35b4-4625-984c-cf3aa7b0f85d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:30.994902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 589/1177] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:19:30 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:19:33.001374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fadf9f-9ff2-4df2-b7f4-033e3fd58b78 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:33.001996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fadf9f-9ff2-4df2-b7f4-033e3fd58b78 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:33.001996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fadf9f-9ff2-4df2-b7f4-033e3fd58b78 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:33.004200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 589/1178] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:33.009648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:33.010045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:33.010206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:33.027897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:33.029123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:33.055310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:33.097136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 590/1179] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:33.369154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fadf9f-9ff2-4df2-b7f4-033e3fd58b78 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:33.468184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fadf9f-9ff2-4df2-b7f4-033e3fd58b78 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:33.502494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:33.503367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:33.503641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:33.503911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:33.513853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:19:33.518789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:33.519931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:19:33.521268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 590/1180] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:33.525870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:33.623024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594016f5-9b56-4d83-97a7-65b9937e399d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:33.656442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:33.657079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:33.657358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:33.657629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:33.658315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:33.658615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:33.659978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:33.667794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 591/1181] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:33.673533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:33.732625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:33.774888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdac750-5f16-4807-b7f9-66b84b96bb55 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:33.808401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:33.809118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:33.809403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:33.809670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] PATH_INFO: `/v3/users/fbe96a5e325643c3898d04e0134341c9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:33.822543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] RBAC: Authorizing `identity:get_user(user_id=fbe96a5e325643c3898d04e0134341c9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:33.823988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:33.825597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 591/1182] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:33 2022] GET /identity/v3/users/fbe96a5e325643c3898d04e0134341c9 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:33.831653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:33.857092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:33.934283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8bdf7-fc9f-406a-848b-9da14f4ff6b8 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:33.969526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:33.970119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:33.970396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:33.971338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:34.041271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:34.042716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:34.066940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:34.141920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 592/1183] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:19:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:19:34.148608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:34.245837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984c3a3-5f2b-4e3d-9736-4738b68753d4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:34.262178 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:34.279779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec2e3d-2875-451d-9840-175bce9fd11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:34.280393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec2e3d-2875-451d-9840-175bce9fd11f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:34.280668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec2e3d-2875-451d-9840-175bce9fd11f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:34.280972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec2e3d-2875-451d-9840-175bce9fd11f None admin] PATH_INFO: `/v3/projects/31ede64442f34e9e9b08393115daa073` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:34.282076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec2e3d-2875-451d-9840-175bce9fd11f None admin] RBAC: Authorizing `identity:get_project(project_id=31ede64442f34e9e9b08393115daa073)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:34.283607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec2e3d-2875-451d-9840-175bce9fd11f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:34.285260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 592/1184] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:34 2022] GET /identity/v3/projects/31ede64442f34e9e9b08393115daa073 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:34.291504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:34.392316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04394706-62eb-40c1-937b-3adbb2ea2908 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:34.415473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:34.427009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:34.427599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:34.427875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:34.428146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:34.428894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=31ede64442f34e9e9b08393115daa073, user.id=fbe96a5e325643c3898d04e0134341c9) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:34.429196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:34.430620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa75335-3ee5-46ce-b92d-1c62391e53c2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:34.435930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 593/1185] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:19:34 2022] GET /identity/v3/role_assignments?user.id=fbe96a5e325643c3898d04e0134341c9&scope.project.id=31ede64442f34e9e9b08393115daa073&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:36.315285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb049aa-9772-4330-8ba0-fa04a40ac851 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:36.315787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb049aa-9772-4330-8ba0-fa04a40ac851 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:36.315787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb049aa-9772-4330-8ba0-fa04a40ac851 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:36.317127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 593/1186] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:19:36.322193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:36.322608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:36.322737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:36.341953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:36.343185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:36.408050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 594/1187] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:19:36.672525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb049aa-9772-4330-8ba0-fa04a40ac851 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:36.770919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb049aa-9772-4330-8ba0-fa04a40ac851 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:36.804090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:36.804684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:36.804978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:36.805241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:36.815544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:19:36.820023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:36.821144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:19:36.822187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 594/1188] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:19:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:36.826985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:36.924666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e06e0e-fa12-4286-8b92-66b811a548e2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:36.942001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:36.959275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:36.959868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:36.960144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:36.960414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:36.961096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:36.961395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:36.962758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:36.971069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 595/1189] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:19:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:36.976951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:37.023718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:37.078292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997988ac-de0b-4bf7-b2eb-761f4c0d2c52 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:37.113390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:37.113998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:37.114273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:37.114542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] PATH_INFO: `/v3/users/fbe96a5e325643c3898d04e0134341c9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:37.115641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] RBAC: Authorizing `identity:get_user(user_id=fbe96a5e325643c3898d04e0134341c9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:37.117092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:37.118652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 595/1190] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:36 2022] GET /identity/v3/users/fbe96a5e325643c3898d04e0134341c9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:37.124416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:37.224680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f4337-d91f-4700-9c8f-9b743610d92a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:37.256878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0705f-5f41-4856-a793-ec728866b2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:37.257479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0705f-5f41-4856-a793-ec728866b2fe None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:37.257750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0705f-5f41-4856-a793-ec728866b2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:37.258015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0705f-5f41-4856-a793-ec728866b2fe None admin] PATH_INFO: `/v3/projects/31ede64442f34e9e9b08393115daa073` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:37.259100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0705f-5f41-4856-a793-ec728866b2fe None admin] RBAC: Authorizing `identity:get_project(project_id=31ede64442f34e9e9b08393115daa073)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:37.260515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0705f-5f41-4856-a793-ec728866b2fe None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:37.262133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 596/1191] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:37 2022] GET /identity/v3/projects/31ede64442f34e9e9b08393115daa073 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:37.268660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:37.371302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c227e-79e6-4fe8-b6f7-0f88555b3e4b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:37.384702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:37.405731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bac287-6c4a-4efc-95a7-709e8ab2304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:37.406378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bac287-6c4a-4efc-95a7-709e8ab2304d None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:37.406652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bac287-6c4a-4efc-95a7-709e8ab2304d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:37.406921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bac287-6c4a-4efc-95a7-709e8ab2304d None admin] PATH_INFO: `/v3/projects/31ede64442f34e9e9b08393115daa073/users/fbe96a5e325643c3898d04e0134341c9/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:37.408846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bac287-6c4a-4efc-95a7-709e8ab2304d None admin] RBAC: Authorizing `identity:create_grant(project_id=31ede64442f34e9e9b08393115daa073, user_id=fbe96a5e325643c3898d04e0134341c9, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:37.410275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bac287-6c4a-4efc-95a7-709e8ab2304d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:37.423801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 596/1192] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:19:37 2022] PUT /identity/v3/projects/31ede64442f34e9e9b08393115daa073/users/fbe96a5e325643c3898d04e0134341c9/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:19:39.300621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262fea-b6f6-4a5c-b946-3b9aa29c4e31 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:39.301142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262fea-b6f6-4a5c-b946-3b9aa29c4e31 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:39.301142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262fea-b6f6-4a5c-b946-3b9aa29c4e31 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:39.302484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 597/1193] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:19:39.307902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:39.308335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:39.308437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:39.326195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:19:39.327361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:19:39.403157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 597/1194] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:19:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:19:39.670745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262fea-b6f6-4a5c-b946-3b9aa29c4e31 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:39.721375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:39.774158 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262fea-b6f6-4a5c-b946-3b9aa29c4e31 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:39.807526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:39.808135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:39.808413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:39.808684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:39.819915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:19:39.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:19:39.825641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:19:39.826483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 598/1195] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:19:39.832402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:39.929811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b30989-d620-4560-ad53-3f1ec815cb6d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:39.931237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:39.965423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:39.966017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:39.966295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:39.966567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:39.967225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:39.967524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:39.968919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:39.976854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 598/1196] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:39.982924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:40.085218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795eb38-f728-45f7-aabe-73cf2bb1d72f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:40.093677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:40.118898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:40.119496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:40.119767 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:40.120036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] PATH_INFO: `/v3/users/fbe96a5e325643c3898d04e0134341c9` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:40.121154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] RBAC: Authorizing `identity:get_user(user_id=fbe96a5e325643c3898d04e0134341c9)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:40.122573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:40.124105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 599/1197] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Aug 24 03:19:39 2022] GET /identity/v3/users/fbe96a5e325643c3898d04e0134341c9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:40.130679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:40.231151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56414f5-60da-425d-a445-6aca11dbe4d3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:40.264845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:40.265433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:40.265707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:40.265976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:40.296394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:40.334302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:40.335749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:40.463396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 599/1198] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:19:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:19:40.469353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:40.568022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377df89-5b8a-4c06-9ae0-a016badbd705 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:40.601290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c7ab7-f05e-4c12-bf9d-db322c7433ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:40.601899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c7ab7-f05e-4c12-bf9d-db322c7433ea None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:40.602177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c7ab7-f05e-4c12-bf9d-db322c7433ea None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:40.602451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c7ab7-f05e-4c12-bf9d-db322c7433ea None admin] PATH_INFO: `/v3/projects/31ede64442f34e9e9b08393115daa073` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:40.603813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c7ab7-f05e-4c12-bf9d-db322c7433ea None admin] RBAC: Authorizing `identity:get_project(project_id=31ede64442f34e9e9b08393115daa073)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:40.605278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c7ab7-f05e-4c12-bf9d-db322c7433ea None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:40.606921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 600/1199] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Aug 24 03:19:40 2022] GET /identity/v3/projects/31ede64442f34e9e9b08393115daa073 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:40.613909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:19:40.673783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:40.710710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d6b1b-044b-41c4-a316-622db60bd881 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:19:40.743519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:19:40.744473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:19:40.744975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:19:40.745437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:19:40.746731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=31ede64442f34e9e9b08393115daa073, user.id=fbe96a5e325643c3898d04e0134341c9) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:19:40.747245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:19:40.749859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbc1b0-155c-4541-90bb-0cbe0637a227 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:19:40.758435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 600/1200] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:19:40 2022] GET /identity/v3/role_assignments?user.id=fbe96a5e325643c3898d04e0134341c9&scope.project.id=31ede64442f34e9e9b08393115daa073&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:19:50.597197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=78 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:19:50.755362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=36 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:36.598430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe61ecc-e401-4cd2-959a-fc5af5ff73b9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:36.599037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe61ecc-e401-4cd2-959a-fc5af5ff73b9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:36.599037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe61ecc-e401-4cd2-959a-fc5af5ff73b9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:36.600744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 601/1201] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:20:36.606743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:36.607253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:36.607357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:36.632144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:20:36.633508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:20:36.731894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 601/1202] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:20:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:20:37.032309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe61ecc-e401-4cd2-959a-fc5af5ff73b9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:37.164370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe61ecc-e401-4cd2-959a-fc5af5ff73b9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:37.205801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:37.206511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:37.206780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:37.207129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:37.221885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:37.223422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:37.242747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:37.244153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:37.245376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:37.245376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:37.245376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:37.246941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 602/1203] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:37.254648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:37.385958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0323b-7022-445e-8620-85ae034cac55 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:37.421490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:37.428653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:37.429655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:37.430238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:37.430544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:37.431264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:37.431586 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:37.433149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011e567-60a0-4c39-8b71-5b2ac7be944b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:37.452872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 602/1204] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:37.454402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:37.524476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:37.586014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f54205-2ea2-458a-b57a-d2015d75882b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:37.629154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878696-19d3-4a94-b459-c929559265e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:37.629982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878696-19d3-4a94-b459-c929559265e5 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:37.630259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878696-19d3-4a94-b459-c929559265e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:37.630536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878696-19d3-4a94-b459-c929559265e5 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:37.631208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9878696-19d3-4a94-b459-c929559265e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:37.632813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9878696-19d3-4a94-b459-c929559265e5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:37.659737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 603/1205] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Aug 24 03:20:37 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:20:39.920923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88104a-6998-49fd-8e32-98785b67057f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:39.921364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88104a-6998-49fd-8e32-98785b67057f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:39.921576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88104a-6998-49fd-8e32-98785b67057f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:39.923042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 603/1206] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:20:39.928362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:39.928823 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:39.928920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:39.949882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:20:39.951136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:20:40.038388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 604/1207] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:20:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:20:40.347456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88104a-6998-49fd-8e32-98785b67057f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:40.476965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88104a-6998-49fd-8e32-98785b67057f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:40.523732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:40.524360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:40.524632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:40.524984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:40.536123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:20:40.540737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:40.541762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:40.542623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 604/1208] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:20:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:40.548319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:40.599219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:40.675987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ff40-b2da-47fd-b461-de5bedcf396e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:40.724219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:40.724998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:40.725313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:40.725586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:40.726643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:40.727145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:40.728940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:40.738805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 605/1209] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:20:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:40.744666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:40.806393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:40.867504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363e487-ac76-4dbb-82c5-8d2aedf5f422 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:40.916922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:40.917813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:40.918138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:40.918422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:40.930134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:40.931690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:40.948666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:40.949813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:40.950732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:40.950732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:40.950732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:40.950732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 605/1210] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:40.955912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:41.072530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:41.075564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7da07-d6f0-4087-90d5-93396635ae05 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:41.113614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:41.114238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:41.114477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:41.114746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:41.115379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:41.115710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:41.117073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:41.130724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 606/1211] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:41.136205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:41.252852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727323de-25d8-4f46-9d0a-8906435bb2d4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:41.254745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:41.295857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:41.296518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:41.296798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:41.297086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:41.305183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:41.306652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:20:41.320215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:41.324229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:41.327717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 606/1212] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:41.329262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:41.457235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544fe0-1081-4d57-a0f3-7e5027c97ea1 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:41.496349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:41.497001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:41.497319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:41.497611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:41.498921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:41.499330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:41.500888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:41.526325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 607/1213] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Aug 24 03:20:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:20:41.527767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:41.652984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e66112-8490-4dd3-bc80-79e280367270 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:41.696298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:41.696977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:41.697254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:41.697519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:41.739559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:41.794479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:41.796905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:41.926502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 607/1214] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:20:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:20:41.933127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:41.955086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:42.055813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33381c-a315-4874-8387-6a4a40222ff9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:42.096478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:42.097092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:42.097427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:42.097774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:42.109716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:42.113752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:42.130848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:42.131865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:42.132808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:42.132808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:42.132808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:42.132808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 608/1215] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:42.137917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:42.207469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:42.250887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a062-f73a-41f3-bda5-1860ac5b4979 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:42.288016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:42.288729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:42.289026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:42.289294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:42.289921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:42.290216 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:42.291571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:42.308539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 608/1216] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:42.309443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:42.392636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:42.434711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2133c-31e5-4572-8dd7-852b6826e8ed None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:42.477349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:42.478084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:42.478353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:42.478617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:42.486795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:42.488361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:20:42.502512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:42.503500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:42.504400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 609/1217] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:20:42.507999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:42.620957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be7094-de18-4b07-b91b-a049ba9c04ce None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:42.645402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:42.659401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:42.660181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:42.660499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:42.660802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:42.661621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:42.661928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:42.663445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6cc5f-a605-4151-88e0-c9d1d282c2be None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:42.680148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 609/1218] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:42.686274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:42.809460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ebc54-b51a-463f-bf55-d5dbbfd9b118 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:42.845183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:42.853157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:42.853784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:42.854066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:42.854338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:42.855096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=bae100a3a8e94ca6b2800df52ab453f6) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:42.855407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:42.857161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989a299-0353-4bbd-a37d-856f8c5697cb None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:42.867196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 610/1219] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:20:42 2022] GET /identity/v3/role_assignments?user.id=bae100a3a8e94ca6b2800df52ab453f6&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:44.793159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c33c2-a6e3-4dbe-9e2c-c3c0e74b81e5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:44.793714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c33c2-a6e3-4dbe-9e2c-c3c0e74b81e5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:44.793806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c33c2-a6e3-4dbe-9e2c-c3c0e74b81e5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:44.795845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 610/1220] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:20:44.802201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:44.802651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:44.802790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:44.827512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:20:44.829217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:20:44.911640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 611/1221] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:20:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:20:45.191444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c33c2-a6e3-4dbe-9e2c-c3c0e74b81e5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:45.298037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c33c2-a6e3-4dbe-9e2c-c3c0e74b81e5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:45.334430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:45.335203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:45.335498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:45.335823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:45.347532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:20:45.353308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:45.354590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:45.355861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 611/1222] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:20:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:45.361121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:45.489123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27323ea7-18b4-4731-a6de-7d0da3983a49 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:45.513187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:45.524346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:45.524973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:45.525249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:45.525515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:45.526173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] RBAC: Adding query filter params (name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:45.526467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:45.527826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:45.536369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 612/1223] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:20:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:45.542327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:45.587640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:45.648002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed97891-6424-4c59-8359-7190a2b0c511 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:45.685331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:45.686550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:45.686826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:45.687131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:45.698917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:45.700551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:45.716433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:45.717717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:45.719291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:45.719291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:45.719291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:45.719291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 612/1224] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:45.723858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:45.845867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09009d49-a34a-4cb6-857f-529cce57b64e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:45.881854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:45.882447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:45.882721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:45.882989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:45.883630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:45.883924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:45.885332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:45.900243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 613/1225] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:45.907268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:46.013767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:46.021361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f68a8-c105-47e1-bbd2-06245602a6aa None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:46.055821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:46.056438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:46.056704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:46.056995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:46.064867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:46.066347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:20:46.077683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:46.078741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:46.079827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 613/1226] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:20:46.084819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:46.141182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:46.197993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4cf1a-00c0-4f86-9726-476509460627 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:46.245554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:46.246224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:46.246497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:46.246762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:46.247562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:46.247867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:46.250012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:46.268079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 614/1227] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Aug 24 03:20:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:46.274325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:46.392640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f20ad-a2dc-4475-8839-6f1854f2c182 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:46.431886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:46.432533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:46.432882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:46.433154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:46.444321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:46.445968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:46.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:46.463290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:46.464401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:46.464401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:46.464401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:46.464401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 614/1228] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:46.469898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:46.582303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3dfbf-0e6b-4aa5-a29e-8fb9dccb3d7f None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:46.585030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:46.618724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:46.619339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:46.619682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:46.619904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:46.620575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:46.620955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:46.622382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:46.635091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 615/1229] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:46.641715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:46.651293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:46.759390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe0d4f-b0be-4258-887a-b36365cb88d0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:46.802811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:46.803437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:46.803714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:46.803986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:46.811617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:46.813133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:20:46.824495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:46.825451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:46.826497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 615/1230] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:46.832163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:46.947972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d40c4b-898e-4e41-b495-bd648b06b3ad None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:46.984692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:46.985412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:46.985772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:46.986106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:46.987027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:46.987367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:46.989199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff7665-9557-4384-978f-f8725cc4c6cf None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:47.003277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 616/1231] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:47.009649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:47.081873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:47.120005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24f065-84cc-4a15-815f-8f72a97c6d5a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:47.159318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6549f4-b3d8-4216-bd2c-d84b6cc65e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:47.160042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6549f4-b3d8-4216-bd2c-d84b6cc65e14 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:47.160260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6549f4-b3d8-4216-bd2c-d84b6cc65e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:47.160529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6549f4-b3d8-4216-bd2c-d84b6cc65e14 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/bae100a3a8e94ca6b2800df52ab453f6/roles/2daf02087587437e8bf574a1cfac52ef` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:47.180663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6549f4-b3d8-4216-bd2c-d84b6cc65e14 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=bae100a3a8e94ca6b2800df52ab453f6, role_id=2daf02087587437e8bf574a1cfac52ef)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:47.182183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6549f4-b3d8-4216-bd2c-d84b6cc65e14 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:47.195487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 616/1232] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:20:47 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/bae100a3a8e94ca6b2800df52ab453f6/roles/2daf02087587437e8bf574a1cfac52ef => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:20:49.191064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43c029-cde3-4070-aa58-e025ed3d81a0 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:49.191576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43c029-cde3-4070-aa58-e025ed3d81a0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:49.191576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43c029-cde3-4070-aa58-e025ed3d81a0 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:49.193050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 617/1233] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:20:49.198679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:49.199164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:49.199273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:49.219206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:20:49.220405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:20:49.293210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 617/1234] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:20:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:20:49.583806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43c029-cde3-4070-aa58-e025ed3d81a0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:49.610834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:49.700721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43c029-cde3-4070-aa58-e025ed3d81a0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:49.739985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:49.740644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:49.740957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:49.741233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:49.752894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:20:49.757593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:49.758760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:49.759787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 618/1235] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Aug 24 03:20:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:49.766445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:49.829328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:49.892197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a51308-91d3-4e25-a486-54a435349c4c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:49.932452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:49.933132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:49.933440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:49.933838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:49.934515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:49.934814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:49.936922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:49.946346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 618/1236] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Aug 24 03:20:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:49.952838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:50.044597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:50.061716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a76ea9-9553-4b23-9a73-edfdd1d3fbac None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:50.103204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:50.104110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:50.104490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:50.104956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:50.116444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:50.118091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:50.133389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:50.134643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:50.135903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:50.135903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:50.135903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:50.135903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 619/1237] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:50.141407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:50.260646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3b617-d939-4096-9567-92e8e5c6aa99 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:50.272889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:50.308793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:50.308793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:50.308793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:50.308793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:50.309291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:50.309291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:50.309291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:50.322900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 619/1238] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:50.330095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:50.456453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b0595-6ba3-43dc-8ccf-e9c26d2cccff None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:50.496645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:50.497345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:50.497619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:50.497885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:50.501580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:50.509769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:50.511670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:20:50.527094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:50.528263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:50.529572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 620/1239] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:50.535256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:50.643333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d8cd4-b571-4343-b298-16a9aef50518 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:50.678502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:50.679314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:50.679667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:50.680040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:50.680906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:50.681314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:50.682855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:50.703333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 620/1240] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Aug 24 03:20:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:50.709293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:50.825356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b9859-3c5a-4e41-977f-3c2eba94687e None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:50.867807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:50.868415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:50.868699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:50.869015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:50.946462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:50.953963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:50.955435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:51.070307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 621/1241] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:20:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:20:51.078599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:51.109889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:51.188421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f48b7-959a-40e5-a6d1-d90282059843 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:51.228484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:51.229143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:51.229437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:51.229689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:51.242386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:51.243794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:51.259617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:51.261317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:51.262521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:51.262521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:51.262521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:51.262521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 621/1242] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:51.267534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:51.377060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:51.384472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260bf5a-a0a5-445c-bb27-d080143aee0b None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:51.424980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:51.425790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:51.426329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:51.426707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:51.427555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:51.427822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:51.429350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:51.443261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 622/1243] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:51.449218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:51.535479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:51.559421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067cd50-1dd9-4731-8bb1-950cae5b709f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:51.598648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:51.599325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:51.599646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:51.599981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:51.607431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:51.609102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:20:51.621153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:51.622277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:51.623280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 622/1244] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:20:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:51.628359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:51.736398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf876e-48a5-4d19-9185-633c928725a7 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:51.771673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:51.772327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:51.772638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:51.772993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:51.773709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:51.774006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:51.775556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934e73a-92bc-4bcd-ad1e-5e64874d790d None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:51.789510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 623/1245] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:20:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:51.795598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:51.906827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a214c2-d071-45c1-878e-c252ecd04865 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:51.944158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:51.945239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:51.945552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:51.945857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:51.946662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] RBAC: Adding query filter params (role.id=2daf02087587437e8bf574a1cfac52ef, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=bae100a3a8e94ca6b2800df52ab453f6) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:51.946991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:51.948533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70847fce-282e-4c64-91d1-8e1d1edf1de9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:51.955768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 623/1246] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:20:51 2022] GET /identity/v3/role_assignments?user.id=bae100a3a8e94ca6b2800df52ab453f6&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=2daf02087587437e8bf574a1cfac52ef => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:53.985313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e93adc-b715-4a14-886e-83cfbb9a1f5e None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:53.985844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e93adc-b715-4a14-886e-83cfbb9a1f5e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:53.985930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e93adc-b715-4a14-886e-83cfbb9a1f5e None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:53.987318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 624/1247] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:20:53.992982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:53.993443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:53.993547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:54.000309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:54.013209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:20:54.015469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:20:54.100270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 624/1248] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:20:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:20:54.380028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e93adc-b715-4a14-886e-83cfbb9a1f5e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:54.401755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:54.496934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e93adc-b715-4a14-886e-83cfbb9a1f5e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:54.535915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:54.536579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:54.536854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:54.537136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:54.549320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:20:54.554126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:54.555326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:54.556166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 625/1249] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:20:54.562331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:54.678130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cc712-ebb5-4302-8040-2404ec621935 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:54.712253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:54.719579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:54.720181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:54.720455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:54.720719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:54.721410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:54.721705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:54.723377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:54.736795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 625/1250] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:54.739551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:54.837051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:54.865526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30eb012-d8cf-41c4-83a0-733f17fa5cbf None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:54.907514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:54.907514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:54.907514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:54.907514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:54.918795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:54.920250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:54.936612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:54.937632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:54.938511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:54.938511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:54.938511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:54.938511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 626/1251] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:54.944667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:55.071130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce48f-9fba-4c70-92e5-f4166e387314 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:55.110166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:55.110766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:55.111040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:55.111305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:55.111933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:55.112225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:55.113620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:55.127077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 626/1252] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:55.133471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:55.240827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11963beb-c5b5-4845-bab3-0e9c3ae22769 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:55.269481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:55.278692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:55.279288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:55.279564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:55.279863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:55.289089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:55.290555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:20:55.302701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:55.303892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:55.305094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 627/1253] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:55.309820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:55.376099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:55.426302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c96a1-5d0d-43d6-87e9-87c9496c1153 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:55.462396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:55.462996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:55.463267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:55.463532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:55.464248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:55.464544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:55.466008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:55.485129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 627/1254] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Aug 24 03:20:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:55.490497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:55.610980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd21d7-2d5d-482c-9737-313b2d5f9cf2 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:55.650662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:55.651288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:55.651597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:55.651837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:55.712257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:55.730562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:55.732336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:55.859449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 628/1255] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:20:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:20:55.868567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:55.977876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ce0bd-7f4e-4021-80ed-73640b0ec7ca None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:55.992613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:56.015425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:56.016048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:56.016341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:56.016585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:56.026983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:56.028607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:56.044415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:56.046013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:56.047323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:56.047323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:56.047323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:56.047323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 628/1256] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:56.051523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:56.149839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:56.163002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3651d-86dc-4400-86ec-f3ce5175a328 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:56.204906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:56.205581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:56.205864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:56.206155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:56.206936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:56.207194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:56.208620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:56.221624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 629/1257] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:56.227708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:56.337779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce667227-b356-4df8-9764-b46b9dec9784 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:56.378819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:56.379504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:56.379857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:56.380140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:56.388021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:56.390435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:20:56.402668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:56.403739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:20:56.404922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 629/1258] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:20:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:56.410232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:56.518229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c940b-37bf-4664-a333-39a809c36030 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:56.553044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:56.553640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:56.553912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:56.554175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:56.554857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:56.555154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:56.556572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf68a3-3417-44cb-8b4a-ba324a004757 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:56.570044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 630/1259] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:20:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:56.576495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:56.606910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:56.694492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9473e4-62a7-400c-807e-223c8c36caca None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:56.732558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:56.733873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:56.734174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:56.734542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:56.735409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=bae100a3a8e94ca6b2800df52ab453f6) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:56.735790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:56.737455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c61617-92f7-4c26-947f-92f701fce8f4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:56.743764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 630/1260] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:20:56 2022] GET /identity/v3/role_assignments?user.id=bae100a3a8e94ca6b2800df52ab453f6&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:58.684494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2427c-bbdc-4417-84dc-8b9e46f28d10 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:58.685100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2427c-bbdc-4417-84dc-8b9e46f28d10 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:58.685100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2427c-bbdc-4417-84dc-8b9e46f28d10 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:58.686405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 631/1261] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:20:58.691612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:58.692026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:58.692150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:58.710300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:20:58.711467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:20:58.776137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 631/1262] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:20:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:20:59.047892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2427c-bbdc-4417-84dc-8b9e46f28d10 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:59.058801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:59.149026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2427c-bbdc-4417-84dc-8b9e46f28d10 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:59.182104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:59.182711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:59.182989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:59.183259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:59.193454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:20:59.198025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:59.199148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:59.200040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 632/1263] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:59.205591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:59.223999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:59.301922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb792ee-6478-499b-90b8-ff0699ac6742 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:59.337388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:59.338167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:59.338430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:59.338748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:59.339491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:59.339833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:59.341418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:59.351277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 632/1264] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:59.358153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:59.429487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:59.463174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029132d0-4751-4d83-a741-4ea3bcb0c900 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:59.496587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:59.497217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:59.497492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:59.497758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:59.508121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:59.509596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:20:59.523066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:59.524056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:20:59.525256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:20:59.525256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:20:59.525256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:59.525256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 633/1265] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:59.529938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:59.618974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:59.637699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3d85a-9605-4300-81b4-9dfd211df320 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:59.670313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:59.671646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:59.671920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:59.672186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:59.672832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:59.673128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:59.674485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:59.685738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 633/1266] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:20:59.691568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:59.790288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4be91f-392a-4eb4-b0ee-6ec02d33c647 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:59.809661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:20:59.824001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:59.824603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:59.824912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:59.825176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:59.832463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:59.833922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:20:59.844588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:20:59.845732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:20:59.846931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 634/1267] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:20:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:20:59.851511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:20:59.959618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ebac-02de-4d43-af0c-b9e4c9e2cd98 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:20:59.993912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:20:59.994503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:20:59.994777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:20:59.995047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:20:59.995759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:20:59.996059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:20:59.997518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:00.007103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:00.016307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 634/1268] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Aug 24 03:20:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:00.022092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:00.121676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39ec3-15c5-41c2-abb9-d5ab43bda641 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:00.155976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:00.156574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:00.156880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:00.157152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:00.167356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:00.168730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:21:00.182813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:00.183908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:21:00.184843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:21:00.184843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:00.184843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:21:00.184843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 635/1269] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:00.190683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:00.287981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1460bec-c3eb-4977-b47e-e303333fc7da None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:00.320629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:00.321261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:00.321532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:00.321798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:00.322420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:00.322716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:00.324086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:00.337327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 635/1270] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:00.343759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:00.377166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:00.445413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d4ba7-1314-4217-81c6-b80e227686e9 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:00.479232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:00.479842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:00.480118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:00.480389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:00.487051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:00.488479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:21:00.499418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:00.500455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:21:00.501608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 636/1271] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:21:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:00.506561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:00.566394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:00.609196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3f43c-68b2-4719-be4f-789f5cfbeeff None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:00.642631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:00.643214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:00.643488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:00.643757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:00.644429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:00.644726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:00.646169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38936a6f-c56a-499d-afa9-e896b330687e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:00.658764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 636/1272] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:21:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:00.664640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:00.755666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:00.765339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5904d5-b3c2-4916-8585-9cbdcb16c820 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:00.799091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32029b-fdf6-4a8b-a65a-d6ae4801cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:00.799721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32029b-fdf6-4a8b-a65a-d6ae4801cca4 None admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:00.799996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32029b-fdf6-4a8b-a65a-d6ae4801cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:00.800265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32029b-fdf6-4a8b-a65a-d6ae4801cca4 None admin] PATH_INFO: `/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/bae100a3a8e94ca6b2800df52ab453f6/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:00.802242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32029b-fdf6-4a8b-a65a-d6ae4801cca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7c18d17b92a4f839c7f6b179443c794, user_id=bae100a3a8e94ca6b2800df52ab453f6, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:00.803680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32029b-fdf6-4a8b-a65a-d6ae4801cca4 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:00.817032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 637/1273] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Aug 24 03:21:00 2022] PUT /identity/v3/projects/b7c18d17b92a4f839c7f6b179443c794/users/bae100a3a8e94ca6b2800df52ab453f6/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:21:02.716244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2965a50-61e6-419a-a5c9-2a6239b906f8 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:02.716849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2965a50-61e6-419a-a5c9-2a6239b906f8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:02.716849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2965a50-61e6-419a-a5c9-2a6239b906f8 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:02.718778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 637/1274] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:02.723959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:02.724378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:02.724495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:02.742923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:02.744102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:02.809177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 638/1275] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:21:03.079818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2965a50-61e6-419a-a5c9-2a6239b906f8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:03.179900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2965a50-61e6-419a-a5c9-2a6239b906f8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:03.197099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:03.214245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:03.214856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:03.215131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:03.215402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:03.225508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:03.230380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:03.231474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:03.232362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 638/1276] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Aug 24 03:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:03.237591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:03.315385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:03.335546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7924678-f487-4d67-a720-53a53d21b4ec None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:03.370042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:03.370629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:03.370903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:03.371167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:03.371815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:03.372108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:03.373497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:03.382089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 639/1277] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:03.387856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:03.487253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6c5f-cad6-400d-bd70-9857422ca7f9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:03.520442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:03.521072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:03.521352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:03.521622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:03.531983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:03.533465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:21:03.547595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:03.548717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:21:03.549882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:21:03.549882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:03.549882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:03.549882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 639/1278] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:03.554624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:03.653669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be5e5-a14e-4f0d-916b-8460c3bb5148 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:03.690715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:03.691311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:03.691589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:03.691863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:03.692505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:03.692829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:03.694195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:03.706325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 640/1279] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:03.712189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:03.743136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:03.812367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186379-71e8-4a8a-8cdd-ef9f095b82d4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:03.847015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:03.847621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:03.847894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:03.848161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:03.855711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:03.857175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:21:03.868720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:03.869864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:03.871062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 640/1280] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:03.876282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:03.880032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:03.976340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65a050-ecaf-476a-a909-dfbd5ce9b6f5 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:04.009944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:04.010536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:04.010814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:04.011086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:04.011807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:04.012109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:04.013557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:04.030964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 641/1281] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Aug 24 03:21:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:04.036828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:04.126595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:04.136618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06518309-f450-4c26-bb05-8b7be4dfbe63 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:04.170248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:04.170784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:04.171645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:04.171922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:04.237702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:04.239124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:04.327720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:04.338805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 641/1282] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Aug 24 03:21:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:21:04.345585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:04.416280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:04.454477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea28e-c71a-4313-abd1-394d917a735c None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:04.488706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:04.489347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:04.489621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:04.489889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:04.500173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:04.501628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:21:04.515527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:04.516632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:21:04.517972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:21:04.517972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:04.517972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:21:04.517972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 642/1283] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:04.522444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:04.620954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf36b-0148-435c-a38e-994ab3714610 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:04.653477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:04.654061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:04.654338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:04.654609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:04.655237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:04.656021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:04.657415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:04.668322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 642/1284] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Aug 24 03:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:04.674636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:04.772237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5053f-ce94-49dc-8100-2ddb0167e95a None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:04.785959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:04.806170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:04.806771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:04.807046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:04.807316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:04.814296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:04.815715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:21:04.826402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:04.828175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: ERROR keystone.server.flask.application Aug 24 03:21:04.829104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 643/1285] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Aug 24 03:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:04.834344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:04.885264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:04.939068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36340b7-7f1c-4007-bb41-b197152c078f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:04.972772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:04.973386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:04.973702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:04.974189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:04.975425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:04.975732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:04.977194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc54a9-d619-44ab-b391-89cbcf25d3fb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:04.995045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 643/1286] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Aug 24 03:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:05.001406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:05.101825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667a84-28f2-49f4-90a7-dbe262eca8e3 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:05.134049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:05.134631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:05.134906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:05.135175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:05.135886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] RBAC: Adding query filter params (role.id=bf40971954c544fd896cd08b9e6e5230, scope.project.id=b7c18d17b92a4f839c7f6b179443c794, user.id=bae100a3a8e94ca6b2800df52ab453f6) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:05.136189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:05.137636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db9d1-b2d0-4aff-a0c1-95b4e62e0c92 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:05.143691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 644/1287] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Aug 24 03:21:05 2022] GET /identity/v3/role_assignments?user.id=bae100a3a8e94ca6b2800df52ab453f6&scope.project.id=b7c18d17b92a4f839c7f6b179443c794&role.id=bf40971954c544fd896cd08b9e6e5230 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:07.057437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808e724-7a71-415f-82c0-b7fbed5657f5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:07.057992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808e724-7a71-415f-82c0-b7fbed5657f5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:07.057992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808e724-7a71-415f-82c0-b7fbed5657f5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:07.059311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 644/1288] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:07.064927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:07.065318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:07.065481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:07.083234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:07.084420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:07.114628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:07.150739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 645/1289] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:21:07.417330 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808e724-7a71-415f-82c0-b7fbed5657f5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:07.518494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808e724-7a71-415f-82c0-b7fbed5657f5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:07.528749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:07.552707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:07.553348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:07.553625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:07.553896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:07.554466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:07.555830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:07.565165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:07.566387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:07.567394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 645/1290] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:21:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:07.572207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:07.674724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b9dc-8801-46cb-be35-52b274d13188 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:07.708907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:07.709503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:07.709784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:07.710055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:07.710680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:07.710983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:07.712349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa97ea-0a3f-4b96-afe6-611d41550916 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:07.717096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 646/1291] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Aug 24 03:21:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:07.722906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:07.822645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af599f88-7173-4ddf-8f9d-38a65dde690b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:07.856376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:07.856985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:07.857256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:07.857516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:07.858124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] RBAC: Adding query filter params (type=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:07.858415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:07.860482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f132-d99c-4634-aeab-b8e77cdf08df None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:07.864742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 646/1292] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Aug 24 03:21:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:09.754899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187da078-02a1-4c17-8350-005d4d1801d9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:09.755427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187da078-02a1-4c17-8350-005d4d1801d9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:09.755427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187da078-02a1-4c17-8350-005d4d1801d9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:09.756792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 647/1293] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:09.761870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c70e1-2050-415f-98a1-c6b29729ae12 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:09.762234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c70e1-2050-415f-98a1-c6b29729ae12 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:09.762421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c70e1-2050-415f-98a1-c6b29729ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:09.781300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-069c70e1-2050-415f-98a1-c6b29729ae12 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:09.782552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-069c70e1-2050-415f-98a1-c6b29729ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:09.811228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:09.848988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 647/1294] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:21:10.121976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187da078-02a1-4c17-8350-005d4d1801d9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:10.152441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:10.224507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187da078-02a1-4c17-8350-005d4d1801d9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:10.257785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a5bdd-e58f-4c4f-b37e-b1ada1858828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:10.258373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a5bdd-e58f-4c4f-b37e-b1ada1858828 None admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:10.258652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a5bdd-e58f-4c4f-b37e-b1ada1858828 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:10.258927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a5bdd-e58f-4c4f-b37e-b1ada1858828 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:10.259575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a5bdd-e58f-4c4f-b37e-b1ada1858828 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:10.260965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a5bdd-e58f-4c4f-b37e-b1ada1858828 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:10.274920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 648/1295] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Aug 24 03:21:10 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:12.305106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a710e-caf6-4124-9dfe-f4e3a19ea1b0 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:12.305628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a710e-caf6-4124-9dfe-f4e3a19ea1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:12.305628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a710e-caf6-4124-9dfe-f4e3a19ea1b0 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:12.306905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 648/1296] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:12.311999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:12.312478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:12.312783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:12.331001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:12.332206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:12.415630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 649/1297] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:21:12.682956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a710e-caf6-4124-9dfe-f4e3a19ea1b0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:12.783771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a710e-caf6-4124-9dfe-f4e3a19ea1b0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:12.816771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:12.817401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:12.817676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:12.817944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:12.818516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:12.819878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:12.829011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:12.830183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:12.831230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 649/1298] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:21:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:12.836019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:12.854710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:12.934492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22585c-ef07-49f1-9ef0-4252b18ab7cd None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:12.970079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:12.970661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:12.970935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:12.971228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:12.971825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:12.972114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:12.973487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:12.978700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 650/1299] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Aug 24 03:21:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:12.984803 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:13.017063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:13.084166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ba47-75cf-4d1a-98d0-d2445b5ec537 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:13.117276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:13.117852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:13.118124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:13.118389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:13.119063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] RBAC: Adding query filter params (interface=public, service_id=b5acc6aa29ad499caf6729ed5d77de75, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:13.119362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:13.120713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791a775-b6b5-4268-8875-16e5c89a043f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:13.125634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 650/1300] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Aug 24 03:21:12 2022] GET /identity/v3/endpoints?service_id=b5acc6aa29ad499caf6729ed5d77de75&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:13.131258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:13.213157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:13.232681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b087a8-b46b-4104-9b67-8fe47fd5ef27 None admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:13.267105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd5d7e-8519-4440-83cd-57829f3b4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:13.267683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd5d7e-8519-4440-83cd-57829f3b4b40 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:13.267956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd5d7e-8519-4440-83cd-57829f3b4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:13.268223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd5d7e-8519-4440-83cd-57829f3b4b40 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:13.268848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd5d7e-8519-4440-83cd-57829f3b4b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:13.270198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd5d7e-8519-4440-83cd-57829f3b4b40 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:13.276841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 651/1301] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Aug 24 03:21:13 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:21:15.165983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55e5b-bd78-4d14-9697-41232c5e6adf None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:15.166497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55e5b-bd78-4d14-9697-41232c5e6adf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:15.166497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f55e5b-bd78-4d14-9697-41232c5e6adf None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:15.167789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 651/1302] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:15.173098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:15.173542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:15.173648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:15.192046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:15.193257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:15.259708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 652/1303] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:21:15.523529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55e5b-bd78-4d14-9697-41232c5e6adf None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:15.618873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:15.626317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f55e5b-bd78-4d14-9697-41232c5e6adf None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:15.660079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:15.660684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:15.660983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:15.661253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:15.661822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:15.663170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:15.671495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:15.677308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Aug 24 03:21:15.678340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 652/1304] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Aug 24 03:21:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:15.679335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:15.754441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:15.777527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28810b8-a6de-485d-97ed-6c463e2b266a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:15.811423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:15.812015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:15.813164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:15.813164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] PATH_INFO: `/v3/services` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:15.813700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:15.813999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:15.815366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c98971-7340-4553-af48-772fb9bf3e54 None admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:15.820358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 653/1305] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Aug 24 03:21:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:15.826183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:15.927372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c808d40-465f-45b3-8491-bc887c460db3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:15.959107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38972d-691c-4c0c-b7c9-bf27efde5787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:15.959691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38972d-691c-4c0c-b7c9-bf27efde5787 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:15.959962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38972d-691c-4c0c-b7c9-bf27efde5787 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:15.960225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38972d-691c-4c0c-b7c9-bf27efde5787 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:15.960893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38972d-691c-4c0c-b7c9-bf27efde5787 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:15.962255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38972d-691c-4c0c-b7c9-bf27efde5787 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:15.981733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 653/1306] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:21:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:25.817411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=30 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:25.977773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:20.620132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550b4dd-9cb7-4891-a8f3-6274b2ce2e40 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.620609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550b4dd-9cb7-4891-a8f3-6274b2ce2e40 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.620691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550b4dd-9cb7-4891-a8f3-6274b2ce2e40 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.624829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 654/1307] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:20.627670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770825c-f1c3-4f97-8b95-cd245c75b863 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.628126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770825c-f1c3-4f97-8b95-cd245c75b863 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.628221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770825c-f1c3-4f97-8b95-cd245c75b863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.773140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9770825c-f1c3-4f97-8b95-cd245c75b863 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:20.777786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9770825c-f1c3-4f97-8b95-cd245c75b863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:20.928857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 654/1308] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:21.225515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a00ab-6281-4b95-9231-0fa8937a1142 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.226426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a00ab-6281-4b95-9231-0fa8937a1142 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.226911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a00ab-6281-4b95-9231-0fa8937a1142 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.228851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 655/1309] 10.222.0.81 () {54 vars in 965 bytes} [Wed Aug 24 03:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:21.235181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5a19-250e-49de-bbd5-c229dc62ec2f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.235471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5a19-250e-49de-bbd5-c229dc62ec2f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.235751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5a19-250e-49de-bbd5-c229dc62ec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.337743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b51a5a19-250e-49de-bbd5-c229dc62ec2f None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:21.339502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b51a5a19-250e-49de-bbd5-c229dc62ec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:21.365106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.432246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 655/1310] 10.222.0.81 () {58 vars in 1062 bytes} [Wed Aug 24 03:22:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:21.441198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a00ab-6281-4b95-9231-0fa8937a1142 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.443152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1e3a00ab-6281-4b95-9231-0fa8937a1142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:21.572701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a00ab-6281-4b95-9231-0fa8937a1142 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.655134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.656548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.657078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.657567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.660178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:21.722387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.784908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:21.787401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34516738-846f-4dcc-a12e-dc560bf9464a service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:21.808902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 656/1311] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:24.069835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b632-dbc0-4a12-a6b7-39b0e0e7b465 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.070353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b632-dbc0-4a12-a6b7-39b0e0e7b465 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.070353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b632-dbc0-4a12-a6b7-39b0e0e7b465 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.071725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 656/1312] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:24.076911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b9c7e-7b21-4db6-8082-5d7faa2d989b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.077329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b9c7e-7b21-4db6-8082-5d7faa2d989b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.077456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b9c7e-7b21-4db6-8082-5d7faa2d989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.163484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-414b9c7e-7b21-4db6-8082-5d7faa2d989b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:24.164645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-414b9c7e-7b21-4db6-8082-5d7faa2d989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:24.268719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 657/1313] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:24.566515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296f2a2-27cc-421b-b611-db12d2bf5c46 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.566793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296f2a2-27cc-421b-b611-db12d2bf5c46 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.567065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296f2a2-27cc-421b-b611-db12d2bf5c46 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.568496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 657/1314] 10.222.0.81 () {54 vars in 965 bytes} [Wed Aug 24 03:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:24.574671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945cc1d-f815-441a-b8e5-5ce4ef25631b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.574950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945cc1d-f815-441a-b8e5-5ce4ef25631b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.575220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945cc1d-f815-441a-b8e5-5ce4ef25631b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.587342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:24.651177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9945cc1d-f815-441a-b8e5-5ce4ef25631b None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:24.652391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9945cc1d-f815-441a-b8e5-5ce4ef25631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:24.728601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 658/1315] 10.222.0.81 () {58 vars in 1062 bytes} [Wed Aug 24 03:22:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:24.737972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296f2a2-27cc-421b-b611-db12d2bf5c46 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:24.739576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f296f2a2-27cc-421b-b611-db12d2bf5c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:24.844012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296f2a2-27cc-421b-b611-db12d2bf5c46 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:24.916151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:24.918913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:24.919714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.920034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.920453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.922269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:25.030051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:25.031491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a561a-7603-4567-9b99-b41b0def42ed service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:25.050406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 658/1316] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:27.487075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c3ac-07d7-4e89-90a1-e67e0e24a7ec None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.487635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c3ac-07d7-4e89-90a1-e67e0e24a7ec None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.487635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c3ac-07d7-4e89-90a1-e67e0e24a7ec None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.489020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 659/1317] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:27.494745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17b0f-98a5-4f77-92d5-e0bc38b4adbb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.495159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17b0f-98a5-4f77-92d5-e0bc38b4adbb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.495270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17b0f-98a5-4f77-92d5-e0bc38b4adbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.560681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-86f17b0f-98a5-4f77-92d5-e0bc38b4adbb None None] MFA Rules not processed for user `043ada6c6c4a47fdac79dc17270a1821`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:27.561917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-86f17b0f-98a5-4f77-92d5-e0bc38b4adbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:27.614596 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:27.660343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 659/1318] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:27.965308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303c3ac-07d7-4e89-90a1-e67e0e24a7ec None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:27.966862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5303c3ac-07d7-4e89-90a1-e67e0e24a7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:28.083084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303c3ac-07d7-4e89-90a1-e67e0e24a7ec None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:28.086858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:28.160819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:28.161339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.161612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.161878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.164116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:28.270588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:28.272529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1a5c7-a5c0-4213-90df-d3628a43aa4d service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:28.291252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 660/1319] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:30.880113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a30a72-e168-4e52-bc80-7e3e43438f95 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:30.880711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a30a72-e168-4e52-bc80-7e3e43438f95 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:30.880711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a30a72-e168-4e52-bc80-7e3e43438f95 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:30.882727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 660/1320] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:30.888191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3aba6-19e0-4298-930a-067d69258c83 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:30.888611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3aba6-19e0-4298-930a-067d69258c83 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:30.888741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3aba6-19e0-4298-930a-067d69258c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:30.931684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:30.957287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-7be3aba6-19e0-4298-930a-067d69258c83 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:30.958486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7be3aba6-19e0-4298-930a-067d69258c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:31.052977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 661/1321] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:31.340099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a30a72-e168-4e52-bc80-7e3e43438f95 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:31.341893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71a30a72-e168-4e52-bc80-7e3e43438f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:31.445294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a30a72-e168-4e52-bc80-7e3e43438f95 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:31.518575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:31.519172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:31.519444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:31.519712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:31.521347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:31.530522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:31.638949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:31.640393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac0af7-f60c-473c-99e8-4d91d6e08c76 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:31.659113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 661/1322] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:34.295613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc769b-a207-45b9-8d60-c279a89ee8ed None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.296153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc769b-a207-45b9-8d60-c279a89ee8ed None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.296257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc769b-a207-45b9-8d60-c279a89ee8ed None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.297594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 662/1323] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:34.303110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869c074-bb6f-4fb2-bd78-c2e588dd001b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.304035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869c074-bb6f-4fb2-bd78-c2e588dd001b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.304256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869c074-bb6f-4fb2-bd78-c2e588dd001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.372756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a869c074-bb6f-4fb2-bd78-c2e588dd001b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:34.373983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a869c074-bb6f-4fb2-bd78-c2e588dd001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:34.434558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.474564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 662/1324] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:34.762624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc769b-a207-45b9-8d60-c279a89ee8ed None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.764141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e2bc769b-a207-45b9-8d60-c279a89ee8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:34.876257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc769b-a207-45b9-8d60-c279a89ee8ed None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.891989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.955179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.955782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.956060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.956330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.957965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:35.069985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:35.071862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedc694-2e46-4b27-a627-73884eef0093 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:35.090420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 663/1325] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:37.282390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a66b4-b264-4678-8064-922543b26642 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.282917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a66b4-b264-4678-8064-922543b26642 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.282917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a66b4-b264-4678-8064-922543b26642 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.284243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 663/1326] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:37.289704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481a02-d777-4ac3-89a0-1cb576b2c791 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.290603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481a02-d777-4ac3-89a0-1cb576b2c791 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.290774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481a02-d777-4ac3-89a0-1cb576b2c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.341674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:37.360633 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f0481a02-d777-4ac3-89a0-1cb576b2c791 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:37.362127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f0481a02-d777-4ac3-89a0-1cb576b2c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:37.462311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 664/1327] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:37.747361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a66b4-b264-4678-8064-922543b26642 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:37.749090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-012a66b4-b264-4678-8064-922543b26642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:37.865148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a66b4-b264-4678-8064-922543b26642 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:37.945064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:37.945664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.945935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.946201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.947777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:37.966077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:38.060517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:38.061930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d5278-6c41-4d7b-8d56-18dc814a3d36 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:38.080524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 664/1328] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:40.060124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d15d-01da-4a1d-9ee5-06592acf0219 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.060722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d15d-01da-4a1d-9ee5-06592acf0219 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.060722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d15d-01da-4a1d-9ee5-06592acf0219 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.062060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 665/1329] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:40.068193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabadaf4-1534-41da-b979-6ae3ba3b38c6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.068726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabadaf4-1534-41da-b979-6ae3ba3b38c6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.068726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabadaf4-1534-41da-b979-6ae3ba3b38c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.135658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-eabadaf4-1534-41da-b979-6ae3ba3b38c6 None None] MFA Rules not processed for user `043ada6c6c4a47fdac79dc17270a1821`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:40.136880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-eabadaf4-1534-41da-b979-6ae3ba3b38c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:40.200797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.234594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 665/1330] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:40.520131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d15d-01da-4a1d-9ee5-06592acf0219 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.521727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c2f2d15d-01da-4a1d-9ee5-06592acf0219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:40.632238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d15d-01da-4a1d-9ee5-06592acf0219 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.651942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.706665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.707194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.707466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.707732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.709328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:40.820120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.821635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59991b-17d3-4a6d-bc0b-76b801054656 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.839903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 666/1331] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:42.982966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330881-228a-40fa-9139-561a2ed6fd87 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.983487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330881-228a-40fa-9139-561a2ed6fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.983487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b330881-228a-40fa-9139-561a2ed6fd87 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:42.984871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 666/1332] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:42.990299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb0f59-fadc-402e-9777-f4db752a92d1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.990670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb0f59-fadc-402e-9777-f4db752a92d1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.990842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb0f59-fadc-402e-9777-f4db752a92d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.055749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:43.058853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5abb0f59-fadc-402e-9777-f4db752a92d1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:43.060541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5abb0f59-fadc-402e-9777-f4db752a92d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:43.156641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 667/1333] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:43.439248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b330881-228a-40fa-9139-561a2ed6fd87 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:43.441112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0b330881-228a-40fa-9139-561a2ed6fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:43.544524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b330881-228a-40fa-9139-561a2ed6fd87 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:43.618354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:43.618936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.619205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.619471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.621074 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:43.648802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:43.736584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:43.738233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc3405-e16b-447b-b540-984bfd34d332 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:43.757263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 667/1334] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:45.725717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0e3a-7a64-4ba3-aeab-d2ed571e1522 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.726252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0e3a-7a64-4ba3-aeab-d2ed571e1522 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.726252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0e3a-7a64-4ba3-aeab-d2ed571e1522 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.727972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 668/1335] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:45.733300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922e59-90cb-4268-a90d-15c6a9574bf7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.733740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922e59-90cb-4268-a90d-15c6a9574bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.733837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922e59-90cb-4268-a90d-15c6a9574bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.799831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a2922e59-90cb-4268-a90d-15c6a9574bf7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:45.801048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a2922e59-90cb-4268-a90d-15c6a9574bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:45.875451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:45.897745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 668/1336] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:46.185034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f0e3a-7a64-4ba3-aeab-d2ed571e1522 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.186567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-315f0e3a-7a64-4ba3-aeab-d2ed571e1522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:46.291963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f0e3a-7a64-4ba3-aeab-d2ed571e1522 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.320127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.366436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.367036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.367838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.368187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.369826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:46.480850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:46.483376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db8519-0f06-45e4-aea4-1df5cf46913a service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:46.502398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 669/1337] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:48.896220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df2541-b65f-42d8-a100-878aaa1c25ea None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:48.896742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df2541-b65f-42d8-a100-878aaa1c25ea None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:48.896891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df2541-b65f-42d8-a100-878aaa1c25ea None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:48.898180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 669/1338] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:48.904129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1b49f-2776-46f8-9bfc-caaf77687ff3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:48.904532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1b49f-2776-46f8-9bfc-caaf77687ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:48.904677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1b49f-2776-46f8-9bfc-caaf77687ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:48.970192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-52e1b49f-2776-46f8-9bfc-caaf77687ff3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:48.971370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-52e1b49f-2776-46f8-9bfc-caaf77687ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:48.979581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:49.067631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 670/1339] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:49.349165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df2541-b65f-42d8-a100-878aaa1c25ea None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:49.350749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-74df2541-b65f-42d8-a100-878aaa1c25ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:49.455860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df2541-b65f-42d8-a100-878aaa1c25ea None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:49.526948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:49.527533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:49.527802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:49.528091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:49.529680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:49.564405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:49.636713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:49.638271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3052997-dac7-4cdc-8e05-7e47721614c7 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:49.657907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 670/1340] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:53.465541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee640e68-f699-4995-bc75-602a40b0fd8d None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.466036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee640e68-f699-4995-bc75-602a40b0fd8d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.466126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee640e68-f699-4995-bc75-602a40b0fd8d None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.467429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 671/1341] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:53.472586 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d4e9-da76-49c7-8e63-9b45803c6a95 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.473069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d4e9-da76-49c7-8e63-9b45803c6a95 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.473173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d4e9-da76-49c7-8e63-9b45803c6a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.540301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5899d4e9-da76-49c7-8e63-9b45803c6a95 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:53.541566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5899d4e9-da76-49c7-8e63-9b45803c6a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:53.626797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:53.639018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 671/1342] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:53.919491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee640e68-f699-4995-bc75-602a40b0fd8d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:53.921117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ee640e68-f699-4995-bc75-602a40b0fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:54.028539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee640e68-f699-4995-bc75-602a40b0fd8d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:54.072599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:54.109943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:54.110561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:54.110835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:54.111101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:54.112670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:54.226658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:54.228464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bc13e-f4a7-467b-8c5e-b7b20e0ee915 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:54.251376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 672/1343] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:22:56.788707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68dd88-2b8c-464f-b108-ae50fedc9b0d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.789280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68dd88-2b8c-464f-b108-ae50fedc9b0d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.789372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68dd88-2b8c-464f-b108-ae50fedc9b0d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.791686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 672/1344] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:56.797566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fad08-7f2f-4d8e-97de-fe1a636d4000 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.798104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fad08-7f2f-4d8e-97de-fe1a636d4000 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.798104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fad08-7f2f-4d8e-97de-fe1a636d4000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.870738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-854fad08-7f2f-4d8e-97de-fe1a636d4000 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:56.872003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-854fad08-7f2f-4d8e-97de-fe1a636d4000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:56.891212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.993588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 673/1345] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:22:57.280099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68dd88-2b8c-464f-b108-ae50fedc9b0d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.281829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-eb68dd88-2b8c-464f-b108-ae50fedc9b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:57.401103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68dd88-2b8c-464f-b108-ae50fedc9b0d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.474707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.475308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.475579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.475842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.477485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:57.522016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.594005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.595369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4f4a4-ead0-4d9b-a388-f00d0b0fc7df service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.614741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 673/1346] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:01.665340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bfb05-3643-42db-84ee-af6e0b4b5dee None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.665835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bfb05-3643-42db-84ee-af6e0b4b5dee None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.666037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bfb05-3643-42db-84ee-af6e0b4b5dee None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.677295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6354c3-6b1c-40b1-8a2c-faf36d3a9fb6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.677295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6354c3-6b1c-40b1-8a2c-faf36d3a9fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.677295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6354c3-6b1c-40b1-8a2c-faf36d3a9fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.678803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 674/1347] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:01 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:01.754461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5a6354c3-6b1c-40b1-8a2c-faf36d3a9fb6 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:01.755708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5a6354c3-6b1c-40b1-8a2c-faf36d3a9fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:01.857573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:01.860407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 674/1348] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:02.152302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bfb05-3643-42db-84ee-af6e0b4b5dee None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:02.153932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-238bfb05-3643-42db-84ee-af6e0b4b5dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:02.264116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bfb05-3643-42db-84ee-af6e0b4b5dee None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:02.321592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:02.352507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:02.352727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:02.353079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:02.353282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:02.355733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:02.477879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:02.480139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead18150-f2e1-4750-8a66-26c407a69051 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:02.500298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 675/1349] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:07.483258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82bfc0-acb0-4366-bc91-230e4b2f9159 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.483797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82bfc0-acb0-4366-bc91-230e4b2f9159 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.483797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82bfc0-acb0-4366-bc91-230e4b2f9159 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.485516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 675/1350] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:07.490763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b8ef-6007-43af-b911-cff87c0357d5 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.491178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b8ef-6007-43af-b911-cff87c0357d5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.491365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1b8ef-6007-43af-b911-cff87c0357d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.604588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-cbd1b8ef-6007-43af-b911-cff87c0357d5 None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:07.606177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-cbd1b8ef-6007-43af-b911-cff87c0357d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:07.626226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.784589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 676/1351] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:11.934182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca7a8-b9c7-4e5c-aa9f-6e29a8a5510e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.934663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca7a8-b9c7-4e5c-aa9f-6e29a8a5510e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.934745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca7a8-b9c7-4e5c-aa9f-6e29a8a5510e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.936242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 676/1352] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:11.941147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c96ef4-070a-47c2-951e-fe6d30f5d8b0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.941565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c96ef4-070a-47c2-951e-fe6d30f5d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.941713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c96ef4-070a-47c2-951e-fe6d30f5d8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.024036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a0c96ef4-070a-47c2-951e-fe6d30f5d8b0 None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:12.025289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a0c96ef4-070a-47c2-951e-fe6d30f5d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:12.100498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.140714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 677/1353] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:13.685459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0311a-574f-4cab-bcc4-939eba39f7d5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.686014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0311a-574f-4cab-bcc4-939eba39f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.686014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0311a-574f-4cab-bcc4-939eba39f7d5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.687497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 677/1354] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:13.692353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140fbf-e71d-40f9-9fa3-15fb196b7f6c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.692803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140fbf-e71d-40f9-9fa3-15fb196b7f6c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.692920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140fbf-e71d-40f9-9fa3-15fb196b7f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.772475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ab140fbf-e71d-40f9-9fa3-15fb196b7f6c None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:13.773718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ab140fbf-e71d-40f9-9fa3-15fb196b7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:13.888963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 678/1355] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:23:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:23:17.071746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349040f-96e4-44f6-8ec1-b6598e5ddf01 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.072291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349040f-96e4-44f6-8ec1-b6598e5ddf01 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.072291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349040f-96e4-44f6-8ec1-b6598e5ddf01 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.073666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 678/1356] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:17.080282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b57416-badb-41b7-87e5-8f338bcf7064 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.080693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b57416-badb-41b7-87e5-8f338bcf7064 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.080871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b57416-badb-41b7-87e5-8f338bcf7064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.130564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.170432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-18b57416-badb-41b7-87e5-8f338bcf7064 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:17.173203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-18b57416-badb-41b7-87e5-8f338bcf7064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:17.305994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 679/1357] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:17.624423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dec03-34ed-418b-bc36-139dc744460e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.626684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dec03-34ed-418b-bc36-139dc744460e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.627944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dec03-34ed-418b-bc36-139dc744460e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.634143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 679/1358] 10.222.0.81 () {54 vars in 962 bytes} [Wed Aug 24 03:23:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:17.642245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad4e9-f211-4d4f-b85a-9d34a9a1598b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.642749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad4e9-f211-4d4f-b85a-9d34a9a1598b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.643246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad4e9-f211-4d4f-b85a-9d34a9a1598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.738016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c82ad4e9-f211-4d4f-b85a-9d34a9a1598b None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:17.739242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c82ad4e9-f211-4d4f-b85a-9d34a9a1598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:17.792135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.872035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 680/1359] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Aug 24 03:23:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:17.879746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dec03-34ed-418b-bc36-139dc744460e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.881281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-550dec03-34ed-418b-bc36-139dc744460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:18.051162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dec03-34ed-418b-bc36-139dc744460e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.130666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.174065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.174666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.174936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.175200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.176808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:18.276981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.279339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39259f77-3533-4e39-b129-024f6dbe6ac2 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.300194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 680/1360] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:20.504554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb3a7-3b8e-4069-ba3f-d55582d8a90b None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.505082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb3a7-3b8e-4069-ba3f-d55582d8a90b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.505082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb3a7-3b8e-4069-ba3f-d55582d8a90b None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.506414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 681/1361] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:20.512322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa123b-39e1-4052-9617-7b3efc93be49 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.512815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa123b-39e1-4052-9617-7b3efc93be49 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.513170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa123b-39e1-4052-9617-7b3efc93be49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.583377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e4fa123b-39e1-4052-9617-7b3efc93be49 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:20.584563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e4fa123b-39e1-4052-9617-7b3efc93be49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:20.628302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:20.703794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 681/1362] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:21.003329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb3a7-3b8e-4069-ba3f-d55582d8a90b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:21.004859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-16ceb3a7-3b8e-4069-ba3f-d55582d8a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:21.156965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb3a7-3b8e-4069-ba3f-d55582d8a90b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:21.166865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:21.282093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:21.282698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.282976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.283248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.284874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:21.388517 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:21.390272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184a902-c7ba-4379-9731-a844c8c52edf service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:21.409849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 682/1363] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:23.717140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb97e5-b9c8-4f14-8dd2-342cd2477b16 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.717727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb97e5-b9c8-4f14-8dd2-342cd2477b16 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.717727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb97e5-b9c8-4f14-8dd2-342cd2477b16 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.719026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 682/1364] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:23.724465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98034f57-3981-4a2d-abe4-e1c00b2e396c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.724896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98034f57-3981-4a2d-abe4-e1c00b2e396c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.725136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98034f57-3981-4a2d-abe4-e1c00b2e396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.742068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:23.797481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-98034f57-3981-4a2d-abe4-e1c00b2e396c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:23.798706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-98034f57-3981-4a2d-abe4-e1c00b2e396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:23.904284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 683/1365] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:24.205490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f91171-7e83-4c35-a084-764c8e1a6644 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:24.205874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f91171-7e83-4c35-a084-764c8e1a6644 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:24.206150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f91171-7e83-4c35-a084-764c8e1a6644 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:24.207625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 683/1366] 10.222.0.81 () {54 vars in 962 bytes} [Wed Aug 24 03:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:24.213236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe6a9a-1d86-47de-9100-392960b9b3cc None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:24.213522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe6a9a-1d86-47de-9100-392960b9b3cc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:24.213795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe6a9a-1d86-47de-9100-392960b9b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:24.304670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-28fe6a9a-1d86-47de-9100-392960b9b3cc None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:24.306304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-28fe6a9a-1d86-47de-9100-392960b9b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:24.347214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:24.438749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 684/1367] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Aug 24 03:23:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:23:24.446528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f91171-7e83-4c35-a084-764c8e1a6644 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:24.448190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-99f91171-7e83-4c35-a084-764c8e1a6644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:24.609999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f91171-7e83-4c35-a084-764c8e1a6644 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:24.632008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:24.731596 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:24.732234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:24.732234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:24.732234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:24.733846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:24.834568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:24.836272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587fb98-487c-48d4-9e0a-2292b329a5e3 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:24.855582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 684/1368] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:25.432674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a36f-de52-432e-a08f-6f59ce4213e9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.433286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a36f-de52-432e-a08f-6f59ce4213e9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.433286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a36f-de52-432e-a08f-6f59ce4213e9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.435588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 685/1369] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:25.450836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d06d7e-4b02-4a75-b491-51fb5dbc6701 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.451110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d06d7e-4b02-4a75-b491-51fb5dbc6701 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.451382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d06d7e-4b02-4a75-b491-51fb5dbc6701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.488591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:25.526920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b2d06d7e-4b02-4a75-b491-51fb5dbc6701 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:25.528815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b2d06d7e-4b02-4a75-b491-51fb5dbc6701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:25.644814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 685/1370] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:26.245834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012bcd3-479a-4cc2-9178-7f7b4f1943e9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.245834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012bcd3-479a-4cc2-9178-7f7b4f1943e9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.245834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012bcd3-479a-4cc2-9178-7f7b4f1943e9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.252829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 686/1371] 10.222.0.81 () {54 vars in 964 bytes} [Wed Aug 24 03:23:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:26.252987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec0e5-07cc-4b87-b170-5ab98f053950 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.252987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec0e5-07cc-4b87-b170-5ab98f053950 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.253485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec0e5-07cc-4b87-b170-5ab98f053950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.343345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-21cec0e5-07cc-4b87-b170-5ab98f053950 None None] MFA Rules not processed for user `06578427361b421fa275befc898a7f72`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:26.344578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-21cec0e5-07cc-4b87-b170-5ab98f053950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:26.396698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.430992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 686/1372] 10.222.0.81 () {58 vars in 1061 bytes} [Wed Aug 24 03:23:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:26.440714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012bcd3-479a-4cc2-9178-7f7b4f1943e9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.442439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3012bcd3-479a-4cc2-9178-7f7b4f1943e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:26.568016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012bcd3-479a-4cc2-9178-7f7b4f1943e9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.619742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.664228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.665014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.665404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.665804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.667850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:26.792358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:26.794430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fcf3f-cdca-4fc3-8181-68e3b4a54f5f service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:26.814922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 687/1373] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:26.884709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d99f-808f-4b43-98c2-668fa06d25e4 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.885023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d99f-808f-4b43-98c2-668fa06d25e4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.885487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d99f-808f-4b43-98c2-668fa06d25e4 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.887289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 687/1374] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:26.893022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef0de-d0eb-4831-b412-3a5d2a7c156e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.893477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef0de-d0eb-4831-b412-3a5d2a7c156e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.893845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef0de-d0eb-4831-b412-3a5d2a7c156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.963524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e63ef0de-d0eb-4831-b412-3a5d2a7c156e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:26.964727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e63ef0de-d0eb-4831-b412-3a5d2a7c156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:26.986685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:27.067167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 688/1375] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:27.375889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1d99f-808f-4b43-98c2-668fa06d25e4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.377593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c1e1d99f-808f-4b43-98c2-668fa06d25e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:27.538308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1d99f-808f-4b43-98c2-668fa06d25e4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.591389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:27.657614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.658469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.658739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.659006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.660580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:27.751451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.753076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46170e-6e60-4b8e-8eca-ec736ef52d41 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.771634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 688/1376] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:29.757744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e620-cbcf-475c-b6b0-249e1a8171ff None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.758256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e620-cbcf-475c-b6b0-249e1a8171ff None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.758352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e620-cbcf-475c-b6b0-249e1a8171ff None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.759640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 689/1377] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:29.765148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09374a5-f6f5-454d-9efa-5687ca1bd2c8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.765558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09374a5-f6f5-454d-9efa-5687ca1bd2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.765678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09374a5-f6f5-454d-9efa-5687ca1bd2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.846486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a09374a5-f6f5-454d-9efa-5687ca1bd2c8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:29.847714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a09374a5-f6f5-454d-9efa-5687ca1bd2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:29.854578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:29.967895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 689/1378] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:30.273446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e620-cbcf-475c-b6b0-249e1a8171ff None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.274961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c3f2e620-cbcf-475c-b6b0-249e1a8171ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:30.482265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e620-cbcf-475c-b6b0-249e1a8171ff None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:30.486736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:30.609405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.610013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.610294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.610564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.612193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:30.709297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.710911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2de8a-ecd4-465e-9780-ac55952e3d0a service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.729944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 690/1379] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:32.719116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db44c-65c7-48b4-a492-84962f36668d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.719679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db44c-65c7-48b4-a492-84962f36668d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.719679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19db44c-65c7-48b4-a492-84962f36668d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.721200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 690/1380] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:32.726426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cf9a8-772a-4c21-b36f-3a4b02b32729 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.726862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cf9a8-772a-4c21-b36f-3a4b02b32729 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.727118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cf9a8-772a-4c21-b36f-3a4b02b32729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.739778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:32.800204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-be4cf9a8-772a-4c21-b36f-3a4b02b32729 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:32.801340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-be4cf9a8-772a-4c21-b36f-3a4b02b32729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:32.908512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 691/1381] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:33.200971 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19db44c-65c7-48b4-a492-84962f36668d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.202719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c19db44c-65c7-48b4-a492-84962f36668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:33.371457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:33.388263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19db44c-65c7-48b4-a492-84962f36668d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.518415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.518948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.519223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.519491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:33.521243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:33.623933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:33.625571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee38b0d-ca8c-4f6e-a663-b81127cd1d3f service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:33.636507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:33.646253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 691/1382] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:35.618207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a599c-c762-4769-9f52-9777bbaef8eb None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.619019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a599c-c762-4769-9f52-9777bbaef8eb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.619019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a599c-c762-4769-9f52-9777bbaef8eb None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.620091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 692/1383] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:35.626634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19105bf-b3ce-4d62-9705-5d04b0b2f7d7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.627092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19105bf-b3ce-4d62-9705-5d04b0b2f7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.627395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19105bf-b3ce-4d62-9705-5d04b0b2f7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.698078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d19105bf-b3ce-4d62-9705-5d04b0b2f7d7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:35.699522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d19105bf-b3ce-4d62-9705-5d04b0b2f7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:35.796882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 692/1384] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:36.082153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a599c-c762-4769-9f52-9777bbaef8eb None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.083678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9f7a599c-c762-4769-9f52-9777bbaef8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:36.170667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.233641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a599c-c762-4769-9f52-9777bbaef8eb None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.362798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.363400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.363686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.363956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.365598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:36.423864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.472910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.475132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2409e-4339-4f6b-89ca-3797658af897 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.496741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 693/1385] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:38.444411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbd90a-0dab-4553-a80b-5cef5508db09 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.444960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbd90a-0dab-4553-a80b-5cef5508db09 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.445083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbd90a-0dab-4553-a80b-5cef5508db09 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.446387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 693/1386] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:38.452721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a8e41-7981-4c65-89ae-af0f3b968e39 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.453250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a8e41-7981-4c65-89ae-af0f3b968e39 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.453250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a8e41-7981-4c65-89ae-af0f3b968e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.528436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-cc3a8e41-7981-4c65-89ae-af0f3b968e39 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:38.529674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-cc3a8e41-7981-4c65-89ae-af0f3b968e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:38.630366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 694/1387] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:38.924275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbd90a-0dab-4553-a80b-5cef5508db09 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.926456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3dbd90a-0dab-4553-a80b-5cef5508db09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:39.006057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:39.077375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbd90a-0dab-4553-a80b-5cef5508db09 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:39.194112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:39.194643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.194918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.195186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.196827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:39.236122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:39.297269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:39.298892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e0e25-decb-4304-aaaf-3b7e00569a1a service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:39.321834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 694/1388] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:41.517290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a190732-0110-4237-bd9d-031d66df6db3 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.517974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a190732-0110-4237-bd9d-031d66df6db3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.517974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a190732-0110-4237-bd9d-031d66df6db3 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.519184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 695/1389] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:41.524807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d72347-27d4-472b-a29d-52c7ec6a79d0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.525229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d72347-27d4-472b-a29d-52c7ec6a79d0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.525370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d72347-27d4-472b-a29d-52c7ec6a79d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.593540 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-15d72347-27d4-472b-a29d-52c7ec6a79d0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:41.594760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-15d72347-27d4-472b-a29d-52c7ec6a79d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:41.695172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 695/1390] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:41.988503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a190732-0110-4237-bd9d-031d66df6db3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.990046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4a190732-0110-4237-bd9d-031d66df6db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:42.022822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.038355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a18dc5-eecd-4665-a116-d82a06a66229 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.038684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a18dc5-eecd-4665-a116-d82a06a66229 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.038945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a18dc5-eecd-4665-a116-d82a06a66229 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.040610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 696/1391] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:42.046244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb64c-34ad-430f-9375-164e0038c3c8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.046518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb64c-34ad-430f-9375-164e0038c3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.046785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb64c-34ad-430f-9375-164e0038c3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.063058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.115339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-23dfb64c-34ad-430f-9375-164e0038c3c8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:42.116820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-23dfb64c-34ad-430f-9375-164e0038c3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:42.143746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a190732-0110-4237-bd9d-031d66df6db3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.218695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 697/1392] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:23:42.247951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.260154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.260934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.261207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.261471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.263040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:42.355303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.356939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f173c6-7cc7-437b-82c7-ea47693e1a93 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.375360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 696/1393] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:42.528291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2662c5-6d1e-44b8-82dc-b9c432b1087b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.528882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2662c5-6d1e-44b8-82dc-b9c432b1087b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.529142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2662c5-6d1e-44b8-82dc-b9c432b1087b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.530626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 698/1394] 10.222.0.81 () {54 vars in 967 bytes} [Wed Aug 24 03:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:42.536159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137958ba-4a70-4f9a-91b2-96abdf236117 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.536813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137958ba-4a70-4f9a-91b2-96abdf236117 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.536813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137958ba-4a70-4f9a-91b2-96abdf236117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.623713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-137958ba-4a70-4f9a-91b2-96abdf236117 None None] MFA Rules not processed for user `c4bd5bc35c41458b9ba20d87738cde16`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:42.624976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-137958ba-4a70-4f9a-91b2-96abdf236117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:42.685004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.710338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 697/1395] 10.222.0.81 () {58 vars in 1064 bytes} [Wed Aug 24 03:23:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:42.718366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2662c5-6d1e-44b8-82dc-b9c432b1087b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.719903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1f2662c5-6d1e-44b8-82dc-b9c432b1087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:42.829655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.836916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2662c5-6d1e-44b8-82dc-b9c432b1087b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.914936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.915545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.916360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.916623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.918708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:43.033120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:43.034576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d9bb0-6095-4a4a-b0fa-bfac078ec849 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:43.053098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 699/1396] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:43.794504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464148a1-ba04-4827-8c52-8d49fd256d19 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:43.794980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464148a1-ba04-4827-8c52-8d49fd256d19 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:43.795326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464148a1-ba04-4827-8c52-8d49fd256d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:43.861593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-464148a1-ba04-4827-8c52-8d49fd256d19 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:43.862796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-464148a1-ba04-4827-8c52-8d49fd256d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:43.953649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 698/1397] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:43.997064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89a9f-45f8-44e0-bc30-1856f97ddf6f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:43.997550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89a9f-45f8-44e0-bc30-1856f97ddf6f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:43.997649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89a9f-45f8-44e0-bc30-1856f97ddf6f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:43.998987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 700/1398] 10.222.0.81 () {54 vars in 943 bytes} [Wed Aug 24 03:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:44.005385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26704b-29e8-4284-b90f-d4d22c08f410 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.005750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26704b-29e8-4284-b90f-d4d22c08f410 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.006097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26704b-29e8-4284-b90f-d4d22c08f410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.082457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2c26704b-29e8-4284-b90f-d4d22c08f410 None None] MFA Rules not processed for user `9d8eb54303a94ec38dbb9209c71381da`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:44.083654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2c26704b-29e8-4284-b90f-d4d22c08f410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:44.096994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:44.194092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 699/1399] 10.222.0.81 () {58 vars in 1040 bytes} [Wed Aug 24 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:44.204071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b89a9f-45f8-44e0-bc30-1856f97ddf6f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:44.205696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-83b89a9f-45f8-44e0-bc30-1856f97ddf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:44.230766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:44.351669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b89a9f-45f8-44e0-bc30-1856f97ddf6f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:44.357091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26230e5-24cb-4501-9e13-ee15343cb761 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.357458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26230e5-24cb-4501-9e13-ee15343cb761 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.357731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26230e5-24cb-4501-9e13-ee15343cb761 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.359115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 700/1400] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:44.364184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.364472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.364745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.434428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:44.435641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:44.459843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:44.460367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.460638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.460933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.463120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:44.470466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:44.515563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:44.538686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 701/1401] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:23:44.551288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:44.552943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4b67e-eda6-44fa-9ab3-964843265e8f service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:44.561122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 701/1402] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:23:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:23:44.863443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:44.864656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:45.003307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a973e-21c6-49c5-8767-77b9be49bffb None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:45.118635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:45.152093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:45.152969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.153249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.153520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.155138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:45.255995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:45.257643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645e28a-6b70-4682-a216-2a089b68b5b3 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:45.281932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 702/1403] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:47.295849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ea294-6391-4678-bc97-70998dc4f19b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.296360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ea294-6391-4678-bc97-70998dc4f19b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.296360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ea294-6391-4678-bc97-70998dc4f19b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.298376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 702/1404] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:47.304130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1de5c7-849b-4137-a90c-39e00928166a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.304596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1de5c7-849b-4137-a90c-39e00928166a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.304825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1de5c7-849b-4137-a90c-39e00928166a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.391353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-de1de5c7-849b-4137-a90c-39e00928166a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:47.392691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-de1de5c7-849b-4137-a90c-39e00928166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:47.451266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:47.509742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 703/1405] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:23:47.802762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ea294-6391-4678-bc97-70998dc4f19b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:47.804269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3c2ea294-6391-4678-bc97-70998dc4f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:47.986154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ea294-6391-4678-bc97-70998dc4f19b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:47.989445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:48.124162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:48.124793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:48.125133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:48.125345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:48.126966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:48.230603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:48.232125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036c991-a785-442a-87dd-62fd3f70d218 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.251827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 703/1406] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:23:50.232887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be293930-5df0-48f1-ad5e-229d2d48a6ed None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.233449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be293930-5df0-48f1-ad5e-229d2d48a6ed None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.233449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be293930-5df0-48f1-ad5e-229d2d48a6ed None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.234762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 704/1407] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:50.241097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fcb44-2831-468d-9320-1c5fb8d42082 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.241508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fcb44-2831-468d-9320-1c5fb8d42082 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.241684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fcb44-2831-468d-9320-1c5fb8d42082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.251724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:50.322817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a28fcb44-2831-468d-9320-1c5fb8d42082 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:50.324058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a28fcb44-2831-468d-9320-1c5fb8d42082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:50.431621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 704/1408] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:23:50.756602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be293930-5df0-48f1-ad5e-229d2d48a6ed None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:50.758413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-be293930-5df0-48f1-ad5e-229d2d48a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:50.941978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be293930-5df0-48f1-ad5e-229d2d48a6ed None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:50.988583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:51.078827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:51.079723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:51.080002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:51.080272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:51.081960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:51.182615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:51.184602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230474f7-b95e-4b7a-be41-f0d9d33c7fbf service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:51.205219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 705/1409] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:00.429550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=62 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:01.202953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=82 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:06.852418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f09c55-2828-4859-a36e-f17d4801d6e3 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.853013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f09c55-2828-4859-a36e-f17d4801d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.853013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f09c55-2828-4859-a36e-f17d4801d6e3 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.854388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 705/1410] 10.222.0.81 () {54 vars in 918 bytes} [Wed Aug 24 03:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:06.859464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fffdfc-56bd-4fc8-9e9c-d8987b09cf6c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.859908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fffdfc-56bd-4fc8-9e9c-d8987b09cf6c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.860035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fffdfc-56bd-4fc8-9e9c-d8987b09cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.936053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-01fffdfc-56bd-4fc8-9e9c-d8987b09cf6c None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:06.937285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-01fffdfc-56bd-4fc8-9e9c-d8987b09cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:07.046460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 706/1411] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Aug 24 03:24:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:07.061759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9c2ed-dcf7-4927-b004-fc73a076a8fe None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:07.062037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9c2ed-dcf7-4927-b004-fc73a076a8fe None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:07.062310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9c2ed-dcf7-4927-b004-fc73a076a8fe None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:07.063636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 706/1412] 10.222.0.81 () {54 vars in 918 bytes} [Wed Aug 24 03:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:07.068355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb98c9-ce6e-4f4c-8d11-d07d07035542 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:07.068641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb98c9-ce6e-4f4c-8d11-d07d07035542 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:07.068946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb98c9-ce6e-4f4c-8d11-d07d07035542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:07.140678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:07.143870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-dbbb98c9-ce6e-4f4c-8d11-d07d07035542 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:07.145649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-dbbb98c9-ce6e-4f4c-8d11-d07d07035542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:07.261628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 707/1413] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Aug 24 03:24:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:09.670026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1620c-bb9d-4304-a687-7c1893f46287 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.670590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1620c-bb9d-4304-a687-7c1893f46287 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.670590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1620c-bb9d-4304-a687-7c1893f46287 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.671896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:09.678059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0a33-ecd1-4627-b431-7815e97956d7 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.678573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0a33-ecd1-4627-b431-7815e97956d7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.678686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0a33-ecd1-4627-b431-7815e97956d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.746636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ca8c0a33-ecd1-4627-b431-7815e97956d7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:09.747841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ca8c0a33-ecd1-4627-b431-7815e97956d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:09.804783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:09.847029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:10.135342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1620c-bb9d-4304-a687-7c1893f46287 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.136894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-12d1620c-bb9d-4304-a687-7c1893f46287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:10.281787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1620c-bb9d-4304-a687-7c1893f46287 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.358365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:10.399139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.399668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.399943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.400209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.401831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:10.491057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.492640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c4ad3-83e9-4004-ada0-6f3d2d783fb5 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.510235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 708/1416] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:11.419313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e91a2-d045-4743-a4e4-162acb1cadf1 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.419847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e91a2-d045-4743-a4e4-162acb1cadf1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.419965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e91a2-d045-4743-a4e4-162acb1cadf1 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.421316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:11.426160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a0484-df97-4593-a31e-17eb38340cd0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.426631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a0484-df97-4593-a31e-17eb38340cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.426736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a0484-df97-4593-a31e-17eb38340cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.502422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-551a0484-df97-4593-a31e-17eb38340cd0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:11.503604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-551a0484-df97-4593-a31e-17eb38340cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:11.529422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.600160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:11.753066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e91a2-d045-4743-a4e4-162acb1cadf1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.754521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b09e91a2-d045-4743-a4e4-162acb1cadf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:11.903845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e91a2-d045-4743-a4e4-162acb1cadf1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.943837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:12.022507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:12.023014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.023288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.023557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.025181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:12.113400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:12.115116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6a8a3-7661-49bb-be03-e51fb08b6826 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:12.133465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 710/1419] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:13.045547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa834-137a-41f2-9f52-3d18163ca62d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.046137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa834-137a-41f2-9f52-3d18163ca62d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.046137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa834-137a-41f2-9f52-3d18163ca62d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.047461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:13.053563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad99b4-ed89-48bf-ad2e-cdc52506b873 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.053980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad99b4-ed89-48bf-ad2e-cdc52506b873 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.054112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad99b4-ed89-48bf-ad2e-cdc52506b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.108514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:13.130856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-aaad99b4-ed89-48bf-ad2e-cdc52506b873 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:13.132243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-aaad99b4-ed89-48bf-ad2e-cdc52506b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:13.230021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:13.385816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa834-137a-41f2-9f52-3d18163ca62d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:13.387420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-804fa834-137a-41f2-9f52-3d18163ca62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:13.534013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa834-137a-41f2-9f52-3d18163ca62d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:13.546746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:13.649909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:13.650513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.650785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.651049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.652653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:13.743803 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:13.745468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e43c6-0bb6-45f2-8c86-b800f21d9fbb service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:13.764141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 711/1422] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:24:14.663046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727da22-2612-476a-b8c5-029c9d807022 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.663606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727da22-2612-476a-b8c5-029c9d807022 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.663606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727da22-2612-476a-b8c5-029c9d807022 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.665048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:14.672338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4f37-5a81-47d3-8d53-f8199759a01f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.672817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4f37-5a81-47d3-8d53-f8199759a01f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.672931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4f37-5a81-47d3-8d53-f8199759a01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.710076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:14.767025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a12a4f37-5a81-47d3-8d53-f8199759a01f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:14.768245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a12a4f37-5a81-47d3-8d53-f8199759a01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:14.868601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:15.017861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727da22-2612-476a-b8c5-029c9d807022 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:15.019364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8727da22-2612-476a-b8c5-029c9d807022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:15.135615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:15.172307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727da22-2612-476a-b8c5-029c9d807022 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:15.285975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:15.286525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:15.286800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:15.287067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:15.288662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:15.372632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:15.387212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:15.388872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457a73b-8ff3-4ed6-bb25-139319ec5ed9 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:15.407603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 713/1425] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:16.281594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660655f-b9f6-4b78-a048-7ed7ebed2763 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.282253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660655f-b9f6-4b78-a048-7ed7ebed2763 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.282253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660655f-b9f6-4b78-a048-7ed7ebed2763 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.283484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:16.288512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19bff9-81a2-4348-b97b-74ee4ced731c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.288924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19bff9-81a2-4348-b97b-74ee4ced731c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.289091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19bff9-81a2-4348-b97b-74ee4ced731c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.366391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bd19bff9-81a2-4348-b97b-74ee4ced731c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:16.367615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bd19bff9-81a2-4348-b97b-74ee4ced731c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:16.464286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:16.615987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660655f-b9f6-4b78-a048-7ed7ebed2763 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.617552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9660655f-b9f6-4b78-a048-7ed7ebed2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:16.714801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:16.779428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660655f-b9f6-4b78-a048-7ed7ebed2763 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:16.899607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:16.900146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.900402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.900725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.902353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:16.952216 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.004141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.005957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e96546-59e5-4be4-8bbc-55bc6ef89899 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.024480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 714/1428] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:17.822302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d0ac2-17b7-4278-b9e0-e74d933ecc64 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.822897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d0ac2-17b7-4278-b9e0-e74d933ecc64 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.822897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d0ac2-17b7-4278-b9e0-e74d933ecc64 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.824197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:17.829248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbb3c2-96fa-49dd-8815-85bbae127ae2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.829663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbb3c2-96fa-49dd-8815-85bbae127ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.829785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbb3c2-96fa-49dd-8815-85bbae127ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.906242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-43dbb3c2-96fa-49dd-8815-85bbae127ae2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:17.907527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-43dbb3c2-96fa-49dd-8815-85bbae127ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:18.005946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:18.162799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d0ac2-17b7-4278-b9e0-e74d933ecc64 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:18.164312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ab4d0ac2-17b7-4278-b9e0-e74d933ecc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:18.214278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:18.313326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d0ac2-17b7-4278-b9e0-e74d933ecc64 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:18.439495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:18.440010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:18.440289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:18.440560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:18.442488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:18.449436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:18.533938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:18.536162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f56d4c-ad54-4275-97f0-de22dfddd068 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:18.553999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 716/1431] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:19.357577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eae38-c733-43f8-9444-bcc258489cef None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.358134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eae38-c733-43f8-9444-bcc258489cef None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.358134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eae38-c733-43f8-9444-bcc258489cef None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.359470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:19.364684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc2934-806a-47c1-a822-470eff53d1f0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.365119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc2934-806a-47c1-a822-470eff53d1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.365263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc2934-806a-47c1-a822-470eff53d1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.440800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-90fc2934-806a-47c1-a822-470eff53d1f0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:19.441956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-90fc2934-806a-47c1-a822-470eff53d1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:19.509925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:19.540373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:19.695405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79eae38-c733-43f8-9444-bcc258489cef None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:19.696892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f79eae38-c733-43f8-9444-bcc258489cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:19.714789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:19.845843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79eae38-c733-43f8-9444-bcc258489cef None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:19.937345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:19.964816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:19.965369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.965641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.965906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.967493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:20.057804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.059406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b63da-2a0c-403a-8150-4cebec0300dc service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.077407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 717/1434] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:20.894222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf7eeb-9940-4306-bbc8-a64cc169ea0a None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.895152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf7eeb-9940-4306-bbc8-a64cc169ea0a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.895432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf7eeb-9940-4306-bbc8-a64cc169ea0a None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.896855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:20.901950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641bd9c-004e-4f7b-bfa7-7a92841e4dc9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.902356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641bd9c-004e-4f7b-bfa7-7a92841e4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.902510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641bd9c-004e-4f7b-bfa7-7a92841e4dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.978652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e641bd9c-004e-4f7b-bfa7-7a92841e4dc9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:20.979867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e641bd9c-004e-4f7b-bfa7-7a92841e4dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:21.021789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.077235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:21.246993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf7eeb-9940-4306-bbc8-a64cc169ea0a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.248484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-abdf7eeb-9940-4306-bbc8-a64cc169ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:21.395761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf7eeb-9940-4306-bbc8-a64cc169ea0a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.450164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.512450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.513080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.513363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.513634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.515289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:21.605296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.606882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0ddde-6479-4a44-84ec-aa47d5affc71 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.626360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 719/1437] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:22.432269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e103f6-2514-4032-b44c-2f65191df4a9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:22.432872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e103f6-2514-4032-b44c-2f65191df4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:22.432872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e103f6-2514-4032-b44c-2f65191df4a9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:22.434203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:22.439557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb54f4-c691-4088-bd3c-364a2e3909dd None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:22.440042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb54f4-c691-4088-bd3c-364a2e3909dd None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:22.440386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb54f4-c691-4088-bd3c-364a2e3909dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:22.518963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-87eb54f4-c691-4088-bd3c-364a2e3909dd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:22.520213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-87eb54f4-c691-4088-bd3c-364a2e3909dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:22.522263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:22.622965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:22.772742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e103f6-2514-4032-b44c-2f65191df4a9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:22.774319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-41e103f6-2514-4032-b44c-2f65191df4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:22.924050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e103f6-2514-4032-b44c-2f65191df4a9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:22.951921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:23.038290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:23.038880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:23.039149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:23.039412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:23.041022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:23.127614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:23.129286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184d0bf-e7c4-42da-867d-9bf21680a5c7 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:23.147238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 720/1440] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:25.013392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fb67-dea5-4be5-87d1-6ab8f227fa19 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:25.013955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fb67-dea5-4be5-87d1-6ab8f227fa19 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:25.013955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fb67-dea5-4be5-87d1-6ab8f227fa19 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:25.015295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:25.021727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168c067-df96-4294-aa51-3fa2340e30ea None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:25.022118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168c067-df96-4294-aa51-3fa2340e30ea None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:25.022276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168c067-df96-4294-aa51-3fa2340e30ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:25.063591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:25.090501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5168c067-df96-4294-aa51-3fa2340e30ea None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:25.091697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5168c067-df96-4294-aa51-3fa2340e30ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:25.195421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:25.491187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fb67-dea5-4be5-87d1-6ab8f227fa19 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:25.492728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bc36fb67-dea5-4be5-87d1-6ab8f227fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:25.598102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fb67-dea5-4be5-87d1-6ab8f227fa19 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:25.631023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:25.673256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:25.673857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:25.674128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:25.674393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:25.676097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:25.786751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:25.788105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5b381-ddac-4809-ba69-113a9ec62ed3 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:25.806256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 722/1443] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:27.897493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a92fa-e3a7-478e-9d27-00bc839069c8 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:27.898007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a92fa-e3a7-478e-9d27-00bc839069c8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:27.898007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a92fa-e3a7-478e-9d27-00bc839069c8 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:27.899322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:27.904648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e31e-054d-44ad-83af-ea58e047ff62 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:27.905060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e31e-054d-44ad-83af-ea58e047ff62 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:27.905229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e31e-054d-44ad-83af-ea58e047ff62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:27.972040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-7c00e31e-054d-44ad-83af-ea58e047ff62 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:27.973373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7c00e31e-054d-44ad-83af-ea58e047ff62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:27.986336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:28.072742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:28.356191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a92fa-e3a7-478e-9d27-00bc839069c8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:28.357806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f64a92fa-e3a7-478e-9d27-00bc839069c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:28.468094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a92fa-e3a7-478e-9d27-00bc839069c8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:28.484247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:28.548353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:28.549014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:28.549294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:28.549562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:28.551663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:28.791140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:28.792593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80bdc3-2ee3-498e-bf77-0e7a08b5dbcd service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:28.811144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 723/1446] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:31.049165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a4c6d-00c6-4ce3-bbd6-dd642fd36ee8 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:31.049788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a4c6d-00c6-4ce3-bbd6-dd642fd36ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:31.049945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a4c6d-00c6-4ce3-bbd6-dd642fd36ee8 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:31.051518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:31.057099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdc3ce-9c53-408e-8243-a1cc3ac1f481 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:31.057525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdc3ce-9c53-408e-8243-a1cc3ac1f481 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:31.057647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdc3ce-9c53-408e-8243-a1cc3ac1f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:31.105811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:31.123888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c1fdc3ce-9c53-408e-8243-a1cc3ac1f481 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:31.125128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c1fdc3ce-9c53-408e-8243-a1cc3ac1f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:31.220171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:31.514343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a4c6d-00c6-4ce3-bbd6-dd642fd36ee8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:31.515909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-639a4c6d-00c6-4ce3-bbd6-dd642fd36ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:31.625524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a4c6d-00c6-4ce3-bbd6-dd642fd36ee8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:31.666299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:31.700891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:31.701487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:31.701763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:31.702030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:31.703622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:31.812824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:31.814307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28512bee-6bcd-41db-a89c-980e8b83afcc service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:31.833530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 725/1449] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:33.845763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be462c57-2062-41d8-85eb-129563469ffb None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:33.846834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be462c57-2062-41d8-85eb-129563469ffb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:33.846834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be462c57-2062-41d8-85eb-129563469ffb None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:33.848325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:33.853584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f087d-a2a4-4451-b1e7-b8c6caaa2d8e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:33.853991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f087d-a2a4-4451-b1e7-b8c6caaa2d8e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:33.854127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f087d-a2a4-4451-b1e7-b8c6caaa2d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:33.923379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ca7f087d-a2a4-4451-b1e7-b8c6caaa2d8e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:33.925636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ca7f087d-a2a4-4451-b1e7-b8c6caaa2d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:33.951332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:34.035276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:34.323111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be462c57-2062-41d8-85eb-129563469ffb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:34.324636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be462c57-2062-41d8-85eb-129563469ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:34.429114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be462c57-2062-41d8-85eb-129563469ffb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:34.448610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:34.502814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:34.503437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:34.503708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:34.503996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:34.505601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:34.614252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:34.615687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e8e06-5d6d-4d1b-8c91-821622d2268f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:34.633531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 726/1452] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:36.603938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0497-85a1-4d2d-a7db-c5b1e58855e0 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:36.604465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0497-85a1-4d2d-a7db-c5b1e58855e0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:36.604465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0497-85a1-4d2d-a7db-c5b1e58855e0 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:36.605938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:36.611430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77051124-d5eb-4567-a11b-6148805dfeba None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:36.612416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77051124-d5eb-4567-a11b-6148805dfeba None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:36.612509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77051124-d5eb-4567-a11b-6148805dfeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:36.676906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:36.680699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-77051124-d5eb-4567-a11b-6148805dfeba None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:36.682346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-77051124-d5eb-4567-a11b-6148805dfeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:36.789901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:37.138197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0497-85a1-4d2d-a7db-c5b1e58855e0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:37.140532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b13e0497-85a1-4d2d-a7db-c5b1e58855e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:37.275297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0497-85a1-4d2d-a7db-c5b1e58855e0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:37.334853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:37.366564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:37.367442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:37.369015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:37.369015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:37.369937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:37.514957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:37.516411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f0518-3917-4fa1-b426-023d39af518f service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:37.538662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 728/1455] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:39.513867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91542ca-33af-4de4-88cd-e2f96c4c68da None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:39.514403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91542ca-33af-4de4-88cd-e2f96c4c68da None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:39.514403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91542ca-33af-4de4-88cd-e2f96c4c68da None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:39.515665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:39.521763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60bc0-4c9d-416f-bffe-6bbcb3d0529b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:39.522150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60bc0-4c9d-416f-bffe-6bbcb3d0529b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:39.522319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60bc0-4c9d-416f-bffe-6bbcb3d0529b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:39.589892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a9b60bc0-4c9d-416f-bffe-6bbcb3d0529b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:39.591118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a9b60bc0-4c9d-416f-bffe-6bbcb3d0529b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:39.620702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:39.690787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:39.974307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91542ca-33af-4de4-88cd-e2f96c4c68da None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:39.975888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f91542ca-33af-4de4-88cd-e2f96c4c68da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:40.077582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91542ca-33af-4de4-88cd-e2f96c4c68da None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:40.105889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:40.151703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:40.152292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:40.152560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:40.152848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:40.154419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:40.264788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:40.266155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9c75e-3915-4788-871a-1bf09bea6e33 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:40.285418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 729/1458] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:42.277313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e06ad-ec2a-480c-b1c5-e54cf31f86a9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:42.277835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e06ad-ec2a-480c-b1c5-e54cf31f86a9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:42.277835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e06ad-ec2a-480c-b1c5-e54cf31f86a9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:42.279173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:42.285046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd0a3-f7a4-4524-ab61-388808009e36 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:42.285480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd0a3-f7a4-4524-ab61-388808009e36 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:42.285576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd0a3-f7a4-4524-ab61-388808009e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:42.353155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-181bd0a3-f7a4-4524-ab61-388808009e36 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:42.354355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-181bd0a3-f7a4-4524-ab61-388808009e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:42.363618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:42.452397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:42.737585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e06ad-ec2a-480c-b1c5-e54cf31f86a9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:42.739121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b03e06ad-ec2a-480c-b1c5-e54cf31f86a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:42.842463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e06ad-ec2a-480c-b1c5-e54cf31f86a9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:42.898965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:42.918702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:42.919310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:42.919590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:42.919862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:42.921486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:43.030924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:43.032287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d221f0-a506-4d32-a814-b7ec2ddfe0bb service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:43.052857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 731/1461] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:45.008476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe72af1-30d4-4cf3-9ab8-2840066ceff3 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:45.009087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe72af1-30d4-4cf3-9ab8-2840066ceff3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:45.009087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe72af1-30d4-4cf3-9ab8-2840066ceff3 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:45.010413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:45.015717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146540c-945f-412b-8437-e04321921ae3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:45.016166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146540c-945f-412b-8437-e04321921ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:45.016271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146540c-945f-412b-8437-e04321921ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:45.208074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-4146540c-945f-412b-8437-e04321921ae3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:45.209218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4146540c-945f-412b-8437-e04321921ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:45.247262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:45.305186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:45.589479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe72af1-30d4-4cf3-9ab8-2840066ceff3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:45.591044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dbe72af1-30d4-4cf3-9ab8-2840066ceff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:45.695705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe72af1-30d4-4cf3-9ab8-2840066ceff3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:45.734782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:45.771495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:45.772093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:45.772366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:45.772675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:45.774375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:45.881196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:45.882713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79061b8-7a17-49ce-8aa1-605cc09f02ac service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:45.903409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 732/1464] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:47.831312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9906d7-cc6a-44fa-96fc-3cb114deeca9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:47.831877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9906d7-cc6a-44fa-96fc-3cb114deeca9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:47.831877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9906d7-cc6a-44fa-96fc-3cb114deeca9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:47.833254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:47.838688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7c235-6e7e-4085-a02a-ada1a561e921 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:47.839119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7c235-6e7e-4085-a02a-ada1a561e921 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:47.839233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7c235-6e7e-4085-a02a-ada1a561e921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:47.904236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b9f7c235-6e7e-4085-a02a-ada1a561e921 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:47.905456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b9f7c235-6e7e-4085-a02a-ada1a561e921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:47.923929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:48.000906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:48.288681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9906d7-cc6a-44fa-96fc-3cb114deeca9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:48.290496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2c9906d7-cc6a-44fa-96fc-3cb114deeca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:48.399025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9906d7-cc6a-44fa-96fc-3cb114deeca9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:48.465038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:48.472743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:48.473363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:48.473636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:48.473902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:48.475466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:48.586580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:48.587928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da9d47-4569-4599-b064-ac6dc6e5a7fe service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:48.606737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 734/1467] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:24:50.550958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055824f-8b7f-4855-8839-8efe5ebe148f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:50.551487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055824f-8b7f-4855-8839-8efe5ebe148f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:50.551487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055824f-8b7f-4855-8839-8efe5ebe148f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:50.552913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:50.558300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412f1d0-86ec-4c34-b9ad-568b4eb3baf2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:50.558758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412f1d0-86ec-4c34-b9ad-568b4eb3baf2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:50.558857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412f1d0-86ec-4c34-b9ad-568b4eb3baf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:50.627880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5412f1d0-86ec-4c34-b9ad-568b4eb3baf2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:50.629129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5412f1d0-86ec-4c34-b9ad-568b4eb3baf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:50.677942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:50.726833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:24:51.013268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055824f-8b7f-4855-8839-8efe5ebe148f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:51.014788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2055824f-8b7f-4855-8839-8efe5ebe148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:51.118173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055824f-8b7f-4855-8839-8efe5ebe148f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:51.165411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:51.192830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:51.193426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:51.193701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:51.193952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:51.195549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:51.309576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:51.311038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a7afe-9c85-4bdd-a17f-66a2be12ccdb service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:51.330584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 735/1470] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:00.725046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=22 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:01.327791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=66 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:11.224417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec1a7-1f12-4b0b-852a-831f124af186 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:11.224956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec1a7-1f12-4b0b-852a-831f124af186 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:11.224956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ec1a7-1f12-4b0b-852a-831f124af186 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:11.226300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 736/1471] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:11.231640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f07558-13a9-4391-a9dc-b6d9ea45df37 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:11.232289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f07558-13a9-4391-a9dc-b6d9ea45df37 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:11.232454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f07558-13a9-4391-a9dc-b6d9ea45df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:11.299130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-77f07558-13a9-4391-a9dc-b6d9ea45df37 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:11.300349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-77f07558-13a9-4391-a9dc-b6d9ea45df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:11.399995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 736/1472] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:11.700909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02a2bd-90a9-4a77-a9fc-a24b0181ef76 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:11.701190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02a2bd-90a9-4a77-a9fc-a24b0181ef76 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:11.701461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02a2bd-90a9-4a77-a9fc-a24b0181ef76 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:11.702860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 737/1473] 10.222.0.81 () {54 vars in 967 bytes} [Wed Aug 24 03:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:11.707807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55023d81-2666-4532-92b1-4ea018d9baff None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:11.708090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55023d81-2666-4532-92b1-4ea018d9baff None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:11.708358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55023d81-2666-4532-92b1-4ea018d9baff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:11.783026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-55023d81-2666-4532-92b1-4ea018d9baff None None] MFA Rules not processed for user `c4bd5bc35c41458b9ba20d87738cde16`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:11.784233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-55023d81-2666-4532-92b1-4ea018d9baff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:11.807018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:11.877418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 737/1474] 10.222.0.81 () {58 vars in 1064 bytes} [Wed Aug 24 03:25:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:11.885203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02a2bd-90a9-4a77-a9fc-a24b0181ef76 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:11.886682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ab02a2bd-90a9-4a77-a9fc-a24b0181ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:11.997592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02a2bd-90a9-4a77-a9fc-a24b0181ef76 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:12.079843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:12.080436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:12.080751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:12.081015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:12.085557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:12.149663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:12.206317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:12.207741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f424350-1d82-461e-9e91-030ea64dde07 service glance] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:12.226316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 738/1475] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:14.306698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b634aa-40ae-46f0-82f9-f272e3a81744 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:14.307250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b634aa-40ae-46f0-82f9-f272e3a81744 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:14.307250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b634aa-40ae-46f0-82f9-f272e3a81744 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:14.308517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 738/1476] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:14.314263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a8d4-21c0-49d5-924d-8287fb26e990 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:14.314677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a8d4-21c0-49d5-924d-8287fb26e990 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:14.314807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a8d4-21c0-49d5-924d-8287fb26e990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:14.384088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e145a8d4-21c0-49d5-924d-8287fb26e990 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:14.385321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e145a8d4-21c0-49d5-924d-8287fb26e990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:14.482694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 739/1477] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:25:14.759161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b634aa-40ae-46f0-82f9-f272e3a81744 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:14.760710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05b634aa-40ae-46f0-82f9-f272e3a81744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:14.893004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b634aa-40ae-46f0-82f9-f272e3a81744 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:14.946912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:14.992980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:14.993579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:14.993853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:14.994125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:14.994759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:14.996471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:25:14.996471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:25:14.996471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:25:14.997643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79c243-3f36-441a-a935-75728e2696a8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:15.016735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 739/1478] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Aug 24 03:25:14 2022] GET /identity/v3/projects => generated 2553 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:25:16.964482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e0fb4-b9ee-4279-86f8-6f4138d68908 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:16.965119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e0fb4-b9ee-4279-86f8-6f4138d68908 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:16.965119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e0fb4-b9ee-4279-86f8-6f4138d68908 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:16.966458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 740/1479] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:16.972637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372761d-2490-48c4-a98a-ca4088400cbc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:16.973116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372761d-2490-48c4-a98a-ca4088400cbc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:16.973226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372761d-2490-48c4-a98a-ca4088400cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:17.040513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d372761d-2490-48c4-a98a-ca4088400cbc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:17.041763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d372761d-2490-48c4-a98a-ca4088400cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:17.138298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 740/1480] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:17.444058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e0fb4-b9ee-4279-86f8-6f4138d68908 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:17.445775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-700e0fb4-b9ee-4279-86f8-6f4138d68908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:17.456823 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:17.601032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e0fb4-b9ee-4279-86f8-6f4138d68908 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:17.682941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:17.716905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:17.717532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:17.717841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:17.718135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:17.719845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:17.815738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:17.818268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a475f0c-fb14-4504-833a-a13f11c6cfef service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:17.838460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 741/1481] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:19.938920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00645da-21d3-4f1c-94ef-0b1b2444b627 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:19.939420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00645da-21d3-4f1c-94ef-0b1b2444b627 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:19.939524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00645da-21d3-4f1c-94ef-0b1b2444b627 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:19.940842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 741/1482] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:19.946052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f85ce-1eb9-4c2d-84d4-bc248be83e69 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:19.946461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f85ce-1eb9-4c2d-84d4-bc248be83e69 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:19.946629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f85ce-1eb9-4c2d-84d4-bc248be83e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:20.013252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5f8f85ce-1eb9-4c2d-84d4-bc248be83e69 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:20.014460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5f8f85ce-1eb9-4c2d-84d4-bc248be83e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:20.063173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:20.115133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 742/1483] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:20.401958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00645da-21d3-4f1c-94ef-0b1b2444b627 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:20.403502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a00645da-21d3-4f1c-94ef-0b1b2444b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:20.428386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:20.508165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00645da-21d3-4f1c-94ef-0b1b2444b627 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:20.583999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:20.584601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:20.584908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:20.585179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:20.586778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:20.660563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:20.694812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:20.696244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292069fb-8d35-48e6-a350-11a629dfd94e service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:20.716181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 742/1484] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:22.691192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a36b-5d75-4d2d-b5da-64a190337b21 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:22.691774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a36b-5d75-4d2d-b5da-64a190337b21 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:22.691774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41a36b-5d75-4d2d-b5da-64a190337b21 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:22.693116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 743/1485] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:22.698224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861a999-935e-416a-9817-459547911c0c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:22.698650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861a999-935e-416a-9817-459547911c0c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:22.698754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861a999-935e-416a-9817-459547911c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:22.765268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d861a999-935e-416a-9817-459547911c0c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:22.766616 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d861a999-935e-416a-9817-459547911c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:22.864387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 743/1486] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:23.153908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a36b-5d75-4d2d-b5da-64a190337b21 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:23.156084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3c41a36b-5d75-4d2d-b5da-64a190337b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:23.269674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41a36b-5d75-4d2d-b5da-64a190337b21 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:23.345589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:23.347497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:23.348367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:23.349098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:23.349596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:23.352280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:23.469227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:23.470576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7f7d1-1fef-4a3c-a88c-a2b2d6fb7500 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:23.488285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 744/1487] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:25.582450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b16f6-d3eb-4b2f-be91-af4e9728ef38 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:25.583113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b16f6-d3eb-4b2f-be91-af4e9728ef38 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:25.583113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b16f6-d3eb-4b2f-be91-af4e9728ef38 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:25.584398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 744/1488] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:25.589999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff814cc9-83bd-401b-871d-4b6b5f3b0145 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:25.590478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff814cc9-83bd-401b-871d-4b6b5f3b0145 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:25.590577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff814cc9-83bd-401b-871d-4b6b5f3b0145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:25.658721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ff814cc9-83bd-401b-871d-4b6b5f3b0145 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:25.659930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ff814cc9-83bd-401b-871d-4b6b5f3b0145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:25.713910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:25.758852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 745/1489] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:26.046357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b16f6-d3eb-4b2f-be91-af4e9728ef38 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:26.047892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a36b16f6-d3eb-4b2f-be91-af4e9728ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:26.083160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:26.152104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b16f6-d3eb-4b2f-be91-af4e9728ef38 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:26.225888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:26.226459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:26.226734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:26.226995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:26.228559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:26.308465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:26.338858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:26.340309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0657a97-1c82-4ef2-b4db-b67d558b7663 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:26.361017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 745/1490] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:28.537244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03113554-6355-4ea0-aaf6-0c11149b4b8a None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:28.537824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03113554-6355-4ea0-aaf6-0c11149b4b8a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:28.537824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03113554-6355-4ea0-aaf6-0c11149b4b8a None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:28.539235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 746/1491] 10.222.0.81 () {54 vars in 927 bytes} [Wed Aug 24 03:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:28.544837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45280-8c0f-4edf-aea3-a390056b68ab None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:28.545229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45280-8c0f-4edf-aea3-a390056b68ab None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:28.545386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45280-8c0f-4edf-aea3-a390056b68ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:28.612975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9bd45280-8c0f-4edf-aea3-a390056b68ab None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:28.614114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9bd45280-8c0f-4edf-aea3-a390056b68ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:28.730121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 746/1492] 10.222.0.81 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:25:29.006822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03113554-6355-4ea0-aaf6-0c11149b4b8a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:29.008340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-03113554-6355-4ea0-aaf6-0c11149b4b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:29.112620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03113554-6355-4ea0-aaf6-0c11149b4b8a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:29.184979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:29.185494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:29.185769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:29.186039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:29.187765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:29.197226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:29.299947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:29.301401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c0717-75fa-4198-be17-4eb65cca21e5 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:29.319570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 747/1493] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:25:38.715652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=82 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:39.317379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=50 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:40.818749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:40.819313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:40.819313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:40.898006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:40.899232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.001030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 747/1494] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:26:41.009618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.010151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.010151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.086515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:41.087626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.183412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 748/1495] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:26:41.189188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:41.190433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.202562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd8c2c-3b25-4fd2-96a1-435be0dd2ee3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:41.265485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:41.303595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:41.304199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.304475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.304747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.305418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:41.305715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:41.306742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:41.306742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:26:41.306742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:26:41.307629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:41.320519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 748/1496] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:41.331223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:41.346264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d02f5d-8f0e-4d9a-ae56-c015ba97ac84 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:41.416058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:41.451870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:41.452669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.452972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.453244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.453876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:41.455609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:41.455609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:26:41.455609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:26:41.456846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:41.514604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 749/1497] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:26:41 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:26:41.518689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:41.520228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.553604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4d20-a72b-4e8c-9412-6664dc5e442d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:41.665367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:41.665986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.666262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.666531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.667152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:41.668162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:41.668162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:26:41.668162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:26:41.669424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:41.713906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 749/1498] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:26:41 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:26:41.718821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:41.720390 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.783167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4de44-b99a-4689-ae10-e92c2db1819e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:41.868455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:41.880133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:41.880733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.881039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.881309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.881956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:41.882995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:41.882995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:26:41.882995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:26:41.883884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:41.896898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 750/1499] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:26:41.901031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:41.902551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.915255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72544482-b799-4b56-ab42-5fab47b51dbc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:41.940963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:42.022466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df561015-5279-451f-a622-c0202950897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:42.023067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df561015-5279-451f-a622-c0202950897d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.023337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df561015-5279-451f-a622-c0202950897d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.023600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df561015-5279-451f-a622-c0202950897d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.024230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df561015-5279-451f-a622-c0202950897d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:42.025267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-df561015-5279-451f-a622-c0202950897d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:42.025267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:26:42.025267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:26:42.026155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df561015-5279-451f-a622-c0202950897d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:42.038686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 750/1500] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:26:42.043025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:42.044524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:42.077780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdb343-f7db-41f2-8001-956a3c71ced3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:42.169872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:42.170574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.170796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.171063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] PATH_INFO: `/v3/projects/54d1ffce1af54023adcafdd976f749e8/users/991d53282cc5499c9d7f84e21d004ea5/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.192688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d1ffce1af54023adcafdd976f749e8, user_id=991d53282cc5499c9d7f84e21d004ea5, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:42.193671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:42.193671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:26:42.193671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:26:42.195614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514056a6-7bda-42c9-a2e5-724602debd13 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:42.221935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 751/1501] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:26:42 2022] PUT /identity/v3/projects/54d1ffce1af54023adcafdd976f749e8/users/991d53282cc5499c9d7f84e21d004ea5/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:26:42.234154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8798-6f8b-4dc8-b5b7-a0c9c1eb8d00 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.234436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8798-6f8b-4dc8-b5b7-a0c9c1eb8d00 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.234725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8798-6f8b-4dc8-b5b7-a0c9c1eb8d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.293141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-578a8798-6f8b-4dc8-b5b7-a0c9c1eb8d00 None None] MFA Rules not processed for user `991d53282cc5499c9d7f84e21d004ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:42.294405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-578a8798-6f8b-4dc8-b5b7-a0c9c1eb8d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:42.370810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 751/1502] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:26:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:26:42.384126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4824-add3-4dae-beac-ddc18fccd023 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.384402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4824-add3-4dae-beac-ddc18fccd023 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.384669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4824-add3-4dae-beac-ddc18fccd023 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.386053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 752/1503] 10.222.0.81 () {54 vars in 964 bytes} [Wed Aug 24 03:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:42.391732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5600f-acd2-4b95-952f-365d68e950c7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.392019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5600f-acd2-4b95-952f-365d68e950c7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.392321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5600f-acd2-4b95-952f-365d68e950c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.424687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:42.471274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8cb5600f-acd2-4b95-952f-365d68e950c7 None None] MFA Rules not processed for user `06578427361b421fa275befc898a7f72`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:42.472399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8cb5600f-acd2-4b95-952f-365d68e950c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:42.547865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 752/1504] 10.222.0.81 () {58 vars in 1061 bytes} [Wed Aug 24 03:26:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:26:42.555969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4824-add3-4dae-beac-ddc18fccd023 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:42.557585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-529d4824-add3-4dae-beac-ddc18fccd023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:42.608650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:42.664464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4824-add3-4dae-beac-ddc18fccd023 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:42.740833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:42.741365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.741645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.741917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.743516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:42.844694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:42.846166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:42.849739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:42.865867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 753/1505] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:26:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:26:42.913487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941c48-9926-4b2c-98a2-4e809cecb8f1 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.913904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941c48-9926-4b2c-98a2-4e809cecb8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.914012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941c48-9926-4b2c-98a2-4e809cecb8f1 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.915255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 753/1506] 10.222.0.81 () {52 vars in 830 bytes} [Wed Aug 24 03:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:42.918926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:42.920210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:43.018989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6a22-a356-4ea6-8a65-0fb9427292ab service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:43.112919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:43.113592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:43.113803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:43.114075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] PATH_INFO: `/v3/users/991d53282cc5499c9d7f84e21d004ea5` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:43.126990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=991d53282cc5499c9d7f84e21d004ea5)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:43.128012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:43.128012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:26:43.128012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:26:43.129270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:43.139047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:43.197052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991d53282cc5499c9d7f84e21d004ea5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:43.211351 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991d53282cc5499c9d7f84e21d004ea5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:43.211796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991d53282cc5499c9d7f84e21d004ea5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:43.220987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991d53282cc5499c9d7f84e21d004ea5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:43.243935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991d53282cc5499c9d7f84e21d004ea5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:43.252111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 754/1507] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:26:42 2022] DELETE /identity/v3/users/991d53282cc5499c9d7f84e21d004ea5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:26:43.256502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:43.256828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:43.257124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:43.315657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:43.316822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:43.394437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:43.408101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 754/1508] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:26:43.417813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:43.419090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:43.520685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14136f-40cd-4c30-b7b4-2ee6a9f0e83a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:43.575533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:43.597156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:43.597758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:43.598035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:43.598304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:43.599887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:43.707732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:43.709206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1b3ad-a9aa-4399-809e-d145d68ae8f7 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:43.727460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 755/1509] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:26:44.057710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:44.059037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:44.070741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f299d-7b62-40ed-9ede-8e690bdeeea8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:44.169783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:44.170435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:44.171383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:44.171649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] PATH_INFO: `/v3/projects/54d1ffce1af54023adcafdd976f749e8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:44.181366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54d1ffce1af54023adcafdd976f749e8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:44.182384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:26:44.182384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:26:44.182384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:26:44.183580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:44.242623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54d1ffce1af54023adcafdd976f749e8', 'request_id': 'req-99419f2b-2527-49d2-8fb7-a616d8077ca6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:44.256341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54d1ffce1af54023adcafdd976f749e8', 'request_id': 'req-99419f2b-2527-49d2-8fb7-a616d8077ca6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:44.265819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54d1ffce1af54023adcafdd976f749e8', 'request_id': 'req-99419f2b-2527-49d2-8fb7-a616d8077ca6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:44.289827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] The token cache is being invalidate because project 54d1ffce1af54023adcafdd976f749e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:26:44.290111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:44.290534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99419f2b-2527-49d2-8fb7-a616d8077ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:26:44.291931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 755/1510] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:26:44 2022] DELETE /identity/v3/projects/54d1ffce1af54023adcafdd976f749e8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:26:53.725165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=62 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:54.287289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:11.394097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98796849-c652-4e61-a613-0ea3a8e103c3 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:11.394699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98796849-c652-4e61-a613-0ea3a8e103c3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:11.394699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98796849-c652-4e61-a613-0ea3a8e103c3 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:11.396014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:11.400978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8124d-2cda-477d-aa8a-768e44bdb9b1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:11.401951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8124d-2cda-477d-aa8a-768e44bdb9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:11.402166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8124d-2cda-477d-aa8a-768e44bdb9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:11.486320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2be8124d-2cda-477d-aa8a-768e44bdb9b1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:11.487921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2be8124d-2cda-477d-aa8a-768e44bdb9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:11.597365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:11.752433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98796849-c652-4e61-a613-0ea3a8e103c3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:11.754564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-98796849-c652-4e61-a613-0ea3a8e103c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:11.873977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98796849-c652-4e61-a613-0ea3a8e103c3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:11.879451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741557a0-89e6-412e-a156-d9754aeec5be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:11.880050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741557a0-89e6-412e-a156-d9754aeec5be service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:11.880329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741557a0-89e6-412e-a156-d9754aeec5be service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:11.880603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741557a0-89e6-412e-a156-d9754aeec5be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:11.889279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741557a0-89e6-412e-a156-d9754aeec5be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:11.890875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741557a0-89e6-412e-a156-d9754aeec5be service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:11.900428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 757/1513] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:12.728618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25651167-13ab-4d1d-af7f-d79a738c25b5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:12.729185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25651167-13ab-4d1d-af7f-d79a738c25b5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:12.729185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25651167-13ab-4d1d-af7f-d79a738c25b5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:12.731489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:12.736282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233cdfa-7641-44f8-8ab9-4a93354dd165 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:12.736690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233cdfa-7641-44f8-8ab9-4a93354dd165 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:12.736851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233cdfa-7641-44f8-8ab9-4a93354dd165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:12.756639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2233cdfa-7641-44f8-8ab9-4a93354dd165 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:12.757851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2233cdfa-7641-44f8-8ab9-4a93354dd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:12.766033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:12.918138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25651167-13ab-4d1d-af7f-d79a738c25b5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:12.932405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25651167-13ab-4d1d-af7f-d79a738c25b5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:12.937988 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f99973-c1b4-453d-9bec-81e084c9d908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:12.938573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f99973-c1b4-453d-9bec-81e084c9d908 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:12.938849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f99973-c1b4-453d-9bec-81e084c9d908 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:12.939125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f99973-c1b4-453d-9bec-81e084c9d908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:12.947482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f99973-c1b4-453d-9bec-81e084c9d908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:12.949115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f99973-c1b4-453d-9bec-81e084c9d908 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:12.958626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 758/1516] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:13.087981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccaacb-d031-4ead-93a2-50b1ab1a94ec None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:13.088285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccaacb-d031-4ead-93a2-50b1ab1a94ec None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:13.088567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccaacb-d031-4ead-93a2-50b1ab1a94ec None None] PATH_INFO: `/v3` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:13.091094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 759/1517] 10.222.0.84 () {54 vars in 925 bytes} [Wed Aug 24 03:28:13 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:28:13.098183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c072f75-bc08-4526-8af1-b15ad53b24b4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:13.098473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c072f75-bc08-4526-8af1-b15ad53b24b4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:13.098750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c072f75-bc08-4526-8af1-b15ad53b24b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:13.159050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3c072f75-bc08-4526-8af1-b15ad53b24b4 None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:13.160244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3c072f75-bc08-4526-8af1-b15ad53b24b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:13.176472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:13.232071 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 759/1518] 10.222.0.84 () {58 vars in 1015 bytes} [Wed Aug 24 03:28:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:22.064826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e770-95ff-4dfb-84ff-2d43a6247d00 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:22.065377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e770-95ff-4dfb-84ff-2d43a6247d00 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:22.065377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e770-95ff-4dfb-84ff-2d43a6247d00 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:22.066639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 760/1519] 10.222.0.81 () {54 vars in 961 bytes} [Wed Aug 24 03:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:22.071556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2c08-bc31-4cfa-814b-8343556fc5e8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:22.071963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2c08-bc31-4cfa-814b-8343556fc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:22.072106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b2c08-bc31-4cfa-814b-8343556fc5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:22.091786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7d7b2c08-bc31-4cfa-814b-8343556fc5e8 None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:22.093830 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7d7b2c08-bc31-4cfa-814b-8343556fc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:22.102260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 760/1520] 10.222.0.81 () {58 vars in 1058 bytes} [Wed Aug 24 03:28:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:22.110503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79e770-95ff-4dfb-84ff-2d43a6247d00 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:22.124605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79e770-95ff-4dfb-84ff-2d43a6247d00 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:22.129613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e93fb-4cc9-44e1-a916-1ac746a4402d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:22.130182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e93fb-4cc9-44e1-a916-1ac746a4402d service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:22.130463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e93fb-4cc9-44e1-a916-1ac746a4402d service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:22.130732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e93fb-4cc9-44e1-a916-1ac746a4402d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:22.138988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e93fb-4cc9-44e1-a916-1ac746a4402d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:22.140614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e93fb-4cc9-44e1-a916-1ac746a4402d service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:22.150093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 761/1521] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:28:24.178415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63daa27-0c18-4ac7-bba9-7cf91060c3ed None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:24.178949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63daa27-0c18-4ac7-bba9-7cf91060c3ed None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:24.178949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63daa27-0c18-4ac7-bba9-7cf91060c3ed None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:24.180291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:24.185149 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a165e0-61b0-41a1-acbc-934000717f4a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:24.185631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a165e0-61b0-41a1-acbc-934000717f4a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:24.185737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a165e0-61b0-41a1-acbc-934000717f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:24.205157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a0a165e0-61b0-41a1-acbc-934000717f4a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:24.206353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a0a165e0-61b0-41a1-acbc-934000717f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:24.214028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:28:24.406906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63daa27-0c18-4ac7-bba9-7cf91060c3ed None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:24.422294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63daa27-0c18-4ac7-bba9-7cf91060c3ed None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:24.427625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9779fa-65d9-401e-824c-0cc2f8c61ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:24.428208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9779fa-65d9-401e-824c-0cc2f8c61ec9 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:24.428529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9779fa-65d9-401e-824c-0cc2f8c61ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:24.428749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9779fa-65d9-401e-824c-0cc2f8c61ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:24.437525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9779fa-65d9-401e-824c-0cc2f8c61ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:24.439191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9779fa-65d9-401e-824c-0cc2f8c61ec9 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:24.448621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 762/1524] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:25.278432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c5300-6b80-4a81-a667-57c213962694 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:25.278958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c5300-6b80-4a81-a667-57c213962694 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:25.278958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c5300-6b80-4a81-a667-57c213962694 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:25.280419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:25.285312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0acb88-f54c-417f-817e-7c24093a6dd4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:25.285774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0acb88-f54c-417f-817e-7c24093a6dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:25.285879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0acb88-f54c-417f-817e-7c24093a6dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:25.305524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9a0acb88-f54c-417f-817e-7c24093a6dd4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:25.306709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9a0acb88-f54c-417f-817e-7c24093a6dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:25.315571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:25.467868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c5300-6b80-4a81-a667-57c213962694 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:25.482054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c5300-6b80-4a81-a667-57c213962694 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:25.487523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd2882-1d24-4784-9aca-2875aed2133f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:25.488357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd2882-1d24-4784-9aca-2875aed2133f service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:25.488655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd2882-1d24-4784-9aca-2875aed2133f service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:25.488955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd2882-1d24-4784-9aca-2875aed2133f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:25.496987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd2882-1d24-4784-9aca-2875aed2133f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:25.499194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd2882-1d24-4784-9aca-2875aed2133f service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:25.508440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 764/1527] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:29.594646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19a4a-025b-44ed-886e-13df098d0816 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:29.595171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19a4a-025b-44ed-886e-13df098d0816 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:29.595171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19a4a-025b-44ed-886e-13df098d0816 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:29.596475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:29.601798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883d438-7b7f-4b07-bc2f-6ac67cd0bde7 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:29.602247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883d438-7b7f-4b07-bc2f-6ac67cd0bde7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:29.602347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883d438-7b7f-4b07-bc2f-6ac67cd0bde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:29.624545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d883d438-7b7f-4b07-bc2f-6ac67cd0bde7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:29.625806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d883d438-7b7f-4b07-bc2f-6ac67cd0bde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:29.633639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:29.944088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19a4a-025b-44ed-886e-13df098d0816 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:29.945853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6ec19a4a-025b-44ed-886e-13df098d0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:30.028612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19a4a-025b-44ed-886e-13df098d0816 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:30.032784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d99157-11cd-4381-b925-2cf190f249c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:30.033302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99157-11cd-4381-b925-2cf190f249c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:30.033759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99157-11cd-4381-b925-2cf190f249c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:30.033759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99157-11cd-4381-b925-2cf190f249c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:30.043244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d99157-11cd-4381-b925-2cf190f249c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:30.044694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d99157-11cd-4381-b925-2cf190f249c9 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:30.054510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 765/1530] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:31.991836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e1c4f-d016-4826-904a-9149b137abc1 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:31.992375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e1c4f-d016-4826-904a-9149b137abc1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:31.992375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e1c4f-d016-4826-904a-9149b137abc1 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:31.993804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:32.000111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536b0e-ca0a-4966-86d2-4b200de65b4e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:32.000528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536b0e-ca0a-4966-86d2-4b200de65b4e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:32.000653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536b0e-ca0a-4966-86d2-4b200de65b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:32.020253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-da536b0e-ca0a-4966-86d2-4b200de65b4e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:32.021479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-da536b0e-ca0a-4966-86d2-4b200de65b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:32.029231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:32.323240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e1c4f-d016-4826-904a-9149b137abc1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:32.324979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-610e1c4f-d016-4826-904a-9149b137abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:32.337949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e1c4f-d016-4826-904a-9149b137abc1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:32.341464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3ccb-9ce4-40d1-972a-942666a619e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:32.342047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3ccb-9ce4-40d1-972a-942666a619e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:32.342841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3ccb-9ce4-40d1-972a-942666a619e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:32.343408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3ccb-9ce4-40d1-972a-942666a619e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:32.351790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f3ccb-9ce4-40d1-972a-942666a619e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:32.353247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f3ccb-9ce4-40d1-972a-942666a619e7 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:32.363155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 767/1533] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:35.865140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72946af4-979e-4abc-8b7b-e79179951ba9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:35.865720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72946af4-979e-4abc-8b7b-e79179951ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:35.865720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72946af4-979e-4abc-8b7b-e79179951ba9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:35.867058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:35.872364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f73e03-9ece-42f8-bd83-950951a28bdf None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:35.872800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f73e03-9ece-42f8-bd83-950951a28bdf None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:35.872950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f73e03-9ece-42f8-bd83-950951a28bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:35.880631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=81 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:35.893909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-37f73e03-9ece-42f8-bd83-950951a28bdf None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:35.895160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-37f73e03-9ece-42f8-bd83-950951a28bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:35.902993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:36.449826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72946af4-979e-4abc-8b7b-e79179951ba9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:36.451482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-72946af4-979e-4abc-8b7b-e79179951ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:36.516552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:36.529990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72946af4-979e-4abc-8b7b-e79179951ba9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:36.533509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d935207-3248-4ed5-bd73-ddc7c192c2c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:36.534097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d935207-3248-4ed5-bd73-ddc7c192c2c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:36.534370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d935207-3248-4ed5-bd73-ddc7c192c2c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:36.534633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d935207-3248-4ed5-bd73-ddc7c192c2c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:36.543311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d935207-3248-4ed5-bd73-ddc7c192c2c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:36.544725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d935207-3248-4ed5-bd73-ddc7c192c2c8 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:36.554644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 768/1536] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:38.474817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d42c8e-968e-4ac7-b23b-8429c6dbc8d3 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:38.475383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d42c8e-968e-4ac7-b23b-8429c6dbc8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:38.475383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d42c8e-968e-4ac7-b23b-8429c6dbc8d3 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:38.477916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:28:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:38.482106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2072e-a588-48f7-a8e0-f767a3c91aa7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:38.482504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2072e-a588-48f7-a8e0-f767a3c91aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:38.482654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2072e-a588-48f7-a8e0-f767a3c91aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:38.502449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e0e2072e-a588-48f7-a8e0-f767a3c91aa7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:38.503659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e0e2072e-a588-48f7-a8e0-f767a3c91aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:38.511511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:39.072826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d42c8e-968e-4ac7-b23b-8429c6dbc8d3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:39.074502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-39d42c8e-968e-4ac7-b23b-8429c6dbc8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:39.088254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d42c8e-968e-4ac7-b23b-8429c6dbc8d3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:39.091849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3696f63-34c7-451b-bd18-e4eade637e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:39.092437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696f63-34c7-451b-bd18-e4eade637e61 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:39.092887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696f63-34c7-451b-bd18-e4eade637e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:39.093169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696f63-34c7-451b-bd18-e4eade637e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:39.102007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3696f63-34c7-451b-bd18-e4eade637e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:39.103429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3696f63-34c7-451b-bd18-e4eade637e61 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:39.113130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 770/1539] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:48.497684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=11 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:49.108588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=8 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:03.490883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.491748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.491748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.518261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:03.519612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:03.531827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 770/1540] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:03.543315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.544353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.544899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.570485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:03.572153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:03.584217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 771/1541] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:03.590046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.617164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47724a-41d5-4219-ab21-338aa6d47695 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.627501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.627868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.628151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.628421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.629115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:03.629555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.631431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.651861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 771/1542] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:03.660635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.671317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9f26-6af2-48ef-8b42-2d292555227a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.681863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.686355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.686355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.686355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.686355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.686355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.757686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 772/1543] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:03 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:03.762507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.779636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc08de7-db13-4b07-9781-2a4527fc2243 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.871132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.871764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.872107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.872383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.873056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.875125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.917633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 772/1544] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:03 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:03.922663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.939021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f0382-095b-4cda-a457-7b6a32f2a6d6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.948254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.949285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.949955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.950445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.951322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.953283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.969679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 773/1545] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:03.975107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.987026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4c571-fbfe-44ac-a793-f09e23d46f67 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.994175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d6312-ace8-4867-aca1-5299ea89ab04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.995129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6312-ace8-4867-aca1-5299ea89ab04 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.995875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6312-ace8-4867-aca1-5299ea89ab04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.996378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6312-ace8-4867-aca1-5299ea89ab04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.997265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d6312-ace8-4867-aca1-5299ea89ab04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.999133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d6312-ace8-4867-aca1-5299ea89ab04 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.015935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 773/1546] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:04.019935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.031553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c65a4-7c8b-4a5b-84ea-78c4bbbacbcd admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.041298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9c847-be3f-43e1-aac3-12c9355d3e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.042338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9c847-be3f-43e1-aac3-12c9355d3e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.042840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9c847-be3f-43e1-aac3-12c9355d3e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.043328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9c847-be3f-43e1-aac3-12c9355d3e69 admin admin] PATH_INFO: `/v3/projects/625e6391470e4018a6c502219b3d2b15/users/33b932f3feb342d889155c3c34407a88/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.060692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5461b3-b03b-4cfb-9ca9-a49fca63b74f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.061369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5461b3-b03b-4cfb-9ca9-a49fca63b74f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.061882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5461b3-b03b-4cfb-9ca9-a49fca63b74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.083322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9c847-be3f-43e1-aac3-12c9355d3e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625e6391470e4018a6c502219b3d2b15, user_id=33b932f3feb342d889155c3c34407a88, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.087246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9c847-be3f-43e1-aac3-12c9355d3e69 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.103740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ff5461b3-b03b-4cfb-9ca9-a49fca63b74f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.105359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ff5461b3-b03b-4cfb-9ca9-a49fca63b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.121494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 774/1547] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:04 2022] PUT /identity/v3/projects/625e6391470e4018a6c502219b3d2b15/users/33b932f3feb342d889155c3c34407a88/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:04.123257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ced6f7-1ae3-4a68-b6c2-ff21a735c65e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.123791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ced6f7-1ae3-4a68-b6c2-ff21a735c65e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.124284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ced6f7-1ae3-4a68-b6c2-ff21a735c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.125350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 774/1548] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.130603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f96b-c280-43e3-a5d2-d85c4734b598 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.131058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f96b-c280-43e3-a5d2-d85c4734b598 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.131338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f96b-c280-43e3-a5d2-d85c4734b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.170451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3dd9f96b-c280-43e3-a5d2-d85c4734b598 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.171783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-91ced6f7-1ae3-4a68-b6c2-ff21a735c65e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.172112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3dd9f96b-c280-43e3-a5d2-d85c4734b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.173451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-91ced6f7-1ae3-4a68-b6c2-ff21a735c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.279239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 775/1549] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:04.280692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ebe81-4e1e-406b-b9d9-c870e59352e1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.281001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ebe81-4e1e-406b-b9d9-c870e59352e1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.281272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ebe81-4e1e-406b-b9d9-c870e59352e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.303533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e19ebe81-4e1e-406b-b9d9-c870e59352e1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.305313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e19ebe81-4e1e-406b-b9d9-c870e59352e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.307815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 775/1550] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.309127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6722a2-34a2-496d-b663-6c20a6eee3e3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.309416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6722a2-34a2-496d-b663-6c20a6eee3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.309692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6722a2-34a2-496d-b663-6c20a6eee3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.317141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 776/1551] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.317141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de20b-c0fa-485c-a776-606e23b3bdee None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.317141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de20b-c0fa-485c-a776-606e23b3bdee None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.317141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de20b-c0fa-485c-a776-606e23b3bdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.331294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ef6722a2-34a2-496d-b663-6c20a6eee3e3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.332506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ef6722a2-34a2-496d-b663-6c20a6eee3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.337910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fc2de20b-c0fa-485c-a776-606e23b3bdee None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.339073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fc2de20b-c0fa-485c-a776-606e23b3bdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.340906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 776/1552] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.342155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.342444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.342714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.354944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 777/1553] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.354944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0d2a0-7181-4bf1-84f4-7f51f6607ff5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.354944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0d2a0-7181-4bf1-84f4-7f51f6607ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.354944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0d2a0-7181-4bf1-84f4-7f51f6607ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.362173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.363369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.369696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4be0d2a0-7181-4bf1-84f4-7f51f6607ff5 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.371126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4be0d2a0-7181-4bf1-84f4-7f51f6607ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.371952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 777/1554] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.372823 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.384458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 778/1555] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.385719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628cc86-4155-4ca9-8b0b-23e8d201ea55 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.386002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628cc86-4155-4ca9-8b0b-23e8d201ea55 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.386273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628cc86-4155-4ca9-8b0b-23e8d201ea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.405880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e628cc86-4155-4ca9-8b0b-23e8d201ea55 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.407065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e628cc86-4155-4ca9-8b0b-23e8d201ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.415824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 779/1556] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.417027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.417310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.417582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.436081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d50b6-1152-435e-9e4f-82aece9ad3e9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.437170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.438361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.439740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a5bec-76ca-43b1-b3ad-dfc0b164783d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.440344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5bec-76ca-43b1-b3ad-dfc0b164783d service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.440625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5bec-76ca-43b1-b3ad-dfc0b164783d service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.440931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5bec-76ca-43b1-b3ad-dfc0b164783d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.446143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 780/1557] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.449928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.450266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a5bec-76ca-43b1-b3ad-dfc0b164783d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.451793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a5bec-76ca-43b1-b3ad-dfc0b164783d service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.456157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d6db6-9b34-4e5e-8bc2-af940b59910d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.460966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.461551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.461823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.462090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.462728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.463023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.463127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 778/1558] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:04.465432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e74392-8164-4e82-a05c-1e53423e7974 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.465432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e74392-8164-4e82-a05c-1e53423e7974 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.465615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8a0a4-de1d-44f2-95ae-a0d84f78cbae admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.465710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e74392-8164-4e82-a05c-1e53423e7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.477706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 781/1559] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.478967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ec30c-2302-4026-813e-b2f10ad2a4d0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.479250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ec30c-2302-4026-813e-b2f10ad2a4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.479531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ec30c-2302-4026-813e-b2f10ad2a4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.487477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-62e74392-8164-4e82-a05c-1e53423e7974 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.488850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-62e74392-8164-4e82-a05c-1e53423e7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.496153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:04.500346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 779/1560] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.501842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10f9d0-1ed1-452d-947e-d5473d1a97ae None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.502130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10f9d0-1ed1-452d-947e-d5473d1a97ae None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.502490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10f9d0-1ed1-452d-947e-d5473d1a97ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.504087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-219ec30c-2302-4026-813e-b2f10ad2a4d0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.506441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-219ec30c-2302-4026-813e-b2f10ad2a4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.515605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 782/1561] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.516863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.517193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.517469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.526611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6d10f9d0-1ed1-452d-947e-d5473d1a97ae None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.527749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6d10f9d0-1ed1-452d-947e-d5473d1a97ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.536644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 780/1562] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.537796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.538248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.539209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.543159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.544599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.551667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:04.554798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 783/1563] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.559455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.571891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce956b4-773e-4e24-94bd-18b25f6208b1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.574324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.575543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.576558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.577192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.577470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.577744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.578385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.578685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.580299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.584253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 781/1564] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:04.588748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.593304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 784/1565] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.594000 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.605444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87d8a4-bf87-4c03-8f1c-2e80b4440cb1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.605744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9a0de-3637-4383-9a16-b17257fca072 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.611010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.611386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.611673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.611806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.611894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.612060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.612247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.612358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.613006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.613124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.613412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.615025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.615739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.631543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 785/1566] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.632813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.644885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945247c4-d561-4dab-ad56-8e36bf694a4c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.651194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.651812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.652352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.652626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.653672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.653978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.655607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.660540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 782/1567] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:04.660540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.669613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 786/1568] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.670765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.685351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928286-5180-494f-8b71-c305a10b80aa admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.747132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708fcc9-8779-4238-b4e1-55333dbf9d15 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.752198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.752826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.753220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.753500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.754151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.754450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.755470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:04.755470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:30:04.755470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:30:04.756349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.760261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.760915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.761203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.761504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.762181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.762484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.764132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.769979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 783/1569] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.770787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.781620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad9340-9bb7-4cc4-8322-bcaf1891d1e8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.784148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 787/1570] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.786516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.787155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.787372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.787661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.788323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:04.789361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.791339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.793380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.803820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95051-612e-431a-94b3-17970189aa5b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.805585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 784/1571] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.806740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.816456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.817485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.817791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.818065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.818475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0bc01-def2-4645-8f67-68f896da99e0 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.818757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.819918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:04.819918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:30:04.819918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:30:04.821228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.823844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.824552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.825047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.825392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.826286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.828640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.850522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 788/1572] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:04.852738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.880050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 785/1573] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:04.884664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.948979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b8f01-b2a4-438d-a2da-96c69a4d42ae admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.961051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd2ba-6c5c-4b99-a408-a95420c218d5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.966697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.967623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.967903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.968176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.968828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.970924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.979926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.005026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 786/1574] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.005725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.014468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.015079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.015355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.015913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.016253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.018539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.032679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.047369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 789/1575] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.048471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.060143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f4c8-0188-41a4-bd37-9ebda4d35694 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.088303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3842d1-8e68-4201-a232-798126e6a01c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.093198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.093873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.094435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.094435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.095118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.097113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.123334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.124095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.124431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.124790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.125493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.127488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.131288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 787/1576] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.131288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.159382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 790/1577] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.160743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.207712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae45811-5969-4749-a2c4-6b22ccb5ef68 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.239227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cb0fe-79c8-4d8d-af20-2850dde8c141 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.244734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.245445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.245992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.246263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.246890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.248860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.273633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 791/1578] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.274493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.287601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cda6f-8e34-4c21-8c2e-b1e92fc9c9b4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.288580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.289735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.290682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.291370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.292086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.293518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.294364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.294751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.294891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.295072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.295800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.297974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.329110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 792/1579] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/users => generated 284 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.329110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.337804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 788/1580] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.338654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.343510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c75d2-9ce8-434e-b144-d1af7c597ea2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.385748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.389665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.426318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcac2b-c505-4ca6-a136-9b799eb52561 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.431777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.432521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.433064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.433254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.433985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.435683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.447368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 789/1581] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.448329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.461350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.462563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.462925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.463273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.464068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.465791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d238-7516-46fc-aa38-2ee614aab87e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.466981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.472477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.473437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.473838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.474520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.475146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.477527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:05.477527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:30:05.477527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:30:05.479296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.501824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 793/1582] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.502974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.511988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 790/1583] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/users => generated 289 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.515540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.517590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f12206-d60b-4ac2-9a93-dfd36173b573 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.524013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.524690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.525027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.525305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.525996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.527243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011de58a-7fb0-4d9c-8ec3-a97d6c083b6a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.528476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.532723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.533527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.533876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.534214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.534988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.536899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.554663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 791/1584] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.554663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.561019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 794/1585] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/users => generated 282 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.562205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.568936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b99b7d-2410-4883-bb32-a057488d9919 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.576353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.577142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.577504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.577843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.578444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2de5a7-852d-4b35-9c43-00d26e2e5916 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.578543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.580236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.584931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.586378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.586732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.587131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.587913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.590134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.594183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 792/1586] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.594410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.608134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9afa5-f1f9-4b6a-a0fc-2a5791f87163 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.617591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.618247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.618526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.618799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.619775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.621527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.625834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 795/1587] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/users => generated 283 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.626612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.639602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 793/1588] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.639602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.639903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582caa55-4eb2-42aa-9b12-ab18b5f961ed admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.645302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.646499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.646499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.646499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.647134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.650060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.651078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0063c-cbd9-436d-82c8-f1402cb8fb2c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.657681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.658295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.658708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.658988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.659682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.661694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.666855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 796/1589] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.668795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.678433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 794/1590] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.679703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.686691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201b350-b8f1-4c12-ba79-bfcc1c5eec8c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.691792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.692482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.692875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.693221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.693940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.696607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.699535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8388-8457-45a5-9fd5-f23c44fba7ae admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.705873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.706762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.707239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.707706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.708543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.708781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 797/1591] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.709781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.712122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.723541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a4613-7b6c-4f17-9e9d-b6604fb07e0b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.729107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 795/1592] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.730536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.731113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.731723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.732000 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.732578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.733425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.735060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.742549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f9830-b715-41f7-88eb-e0e69bd948c8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.747903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.748747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.749258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.750111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.750947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.752177 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 798/1593] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.753065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.753440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.766125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d318-594f-4532-89c8-cf9b9d4c5c4f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.768551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 796/1594] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.769598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.772398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.774471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.774865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.775234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] PATH_INFO: `/v3/projects/84fe38d4a2f24b5ba577c169f125bf67/users/e09d29c12c6e4dd2a0e089f18252cda5/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.784118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116cf50-cac2-40a8-b178-d8a2b2026f41 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.789208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.790134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.790625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.791179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.792659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.794506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.801478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84fe38d4a2f24b5ba577c169f125bf67, user_id=e09d29c12c6e4dd2a0e089f18252cda5, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.802884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:05.802884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:30:05.802884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:30:05.806158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.809618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 797/1595] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.810652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.825643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464a4a-80f5-4f0d-b1c2-54d73248d5b9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.826578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 799/1596] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:05 2022] PUT /identity/v3/projects/84fe38d4a2f24b5ba577c169f125bf67/users/e09d29c12c6e4dd2a0e089f18252cda5/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:05.827438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.908716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1e9d-2150-4631-bca0-31b76c7e4423 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.911334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.912969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.913392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.913674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.914464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.915050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.915923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.916284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.916597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.917088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] PATH_INFO: `/v3/projects/6f655a873ed64b69ac5108f587f8e040/users/e3f543c8ec9f4cdcb7d1dfa0b37c094f/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.932934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 798/1597] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.934171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.938352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f655a873ed64b69ac5108f587f8e040, user_id=e3f543c8ec9f4cdcb7d1dfa0b37c094f, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.942073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.947249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2469-0ace-4f36-b01b-b5bb1c5074de admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.953487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.954227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.954548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.954770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.955666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.957592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.957986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.968837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 800/1598] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:05 2022] PUT /identity/v3/projects/6f655a873ed64b69ac5108f587f8e040/users/e3f543c8ec9f4cdcb7d1dfa0b37c094f/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:05.981730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.987367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 799/1599] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:05.989969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.016721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffee5f0-309c-4837-8df1-0f64f038d2ed admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.053111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:06.079851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152bd4d-b574-4cc6-beea-8fff37b66a2b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.086118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.086997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.087278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.087550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] PATH_INFO: `/v3/projects/cf053bc966194447b1f624d379f1ff72/users/052a827bf2ae437b851572340b67c370/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.111549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf053bc966194447b1f624d379f1ff72, user_id=052a827bf2ae437b851572340b67c370, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.112339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb73482-45f8-478c-aa5b-f71e63b2b71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.113412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb73482-45f8-478c-aa5b-f71e63b2b71d admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.113793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb73482-45f8-478c-aa5b-f71e63b2b71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.114162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb73482-45f8-478c-aa5b-f71e63b2b71d admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.114369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.115386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb73482-45f8-478c-aa5b-f71e63b2b71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.120064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb73482-45f8-478c-aa5b-f71e63b2b71d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.136943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 801/1600] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:05 2022] PUT /identity/v3/projects/cf053bc966194447b1f624d379f1ff72/users/052a827bf2ae437b851572340b67c370/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:06.136943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.137228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 800/1601] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:06.138680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.139009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.139428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.177859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] MFA Rules not processed for user `e09d29c12c6e4dd2a0e089f18252cda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:06.180493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:06.225609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b14a8-a93f-4ced-bfde-a74a014e2945 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.238153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfa48-8b6d-423d-bbfe-1c90dc60800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.238829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfa48-8b6d-423d-bbfe-1c90dc60800c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.239108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfa48-8b6d-423d-bbfe-1c90dc60800c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.239384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfa48-8b6d-423d-bbfe-1c90dc60800c admin admin] PATH_INFO: `/v3/projects/0be0cbafba614105869a0913a21a127b/users/ce4bf5ffbf2d47c9b54c4c1e343e8944/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.267317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfa48-8b6d-423d-bbfe-1c90dc60800c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be0cbafba614105869a0913a21a127b, user_id=ce4bf5ffbf2d47c9b54c4c1e343e8944, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.269270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfa48-8b6d-423d-bbfe-1c90dc60800c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.372722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 801/1602] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:06.372722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.373028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 802/1603] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:06 2022] PUT /identity/v3/projects/0be0cbafba614105869a0913a21a127b/users/ce4bf5ffbf2d47c9b54c4c1e343e8944/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:06.373028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.373028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.373028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.373853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0e153-8aff-4851-8dc6-dd64a969ec37 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.373965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] MFA Rules not processed for user `e3f543c8ec9f4cdcb7d1dfa0b37c094f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:06.373965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:06.453522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd91baf-addf-473a-933e-fe22aef0dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.454353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91baf-addf-473a-933e-fe22aef0dc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.454738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91baf-addf-473a-933e-fe22aef0dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.455016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91baf-addf-473a-933e-fe22aef0dc42 admin admin] PATH_INFO: `/v3/projects/6a1e9994c1c045c892059d5b80578ba7/users/66900b813b934514a7920ba975a05ec5/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.471631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:06.476127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd91baf-addf-473a-933e-fe22aef0dc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1e9994c1c045c892059d5b80578ba7, user_id=66900b813b934514a7920ba975a05ec5, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.480669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd91baf-addf-473a-933e-fe22aef0dc42 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.492920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 803/1604] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:06.494151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.512563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 802/1605] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:06 2022] PUT /identity/v3/projects/6a1e9994c1c045c892059d5b80578ba7/users/66900b813b934514a7920ba975a05ec5/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:06.514081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.514364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.514635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.528627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d489e-18c2-4bce-9633-a8ce37ff725e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.592589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] MFA Rules not processed for user `052a827bf2ae437b851572340b67c370`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:06.599863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:06.664014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:06.704679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06595ee-4573-4807-b594-8872d24a8543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.705497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06595ee-4573-4807-b594-8872d24a8543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.705771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06595ee-4573-4807-b594-8872d24a8543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.706219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06595ee-4573-4807-b594-8872d24a8543 admin admin] PATH_INFO: `/v3/projects/98c3e8f78b614fa9a55f1a677a5b0d81/users/80704e1de7ae4cf49a3deb166c4e82c9/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.733193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06595ee-4573-4807-b594-8872d24a8543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c3e8f78b614fa9a55f1a677a5b0d81, user_id=80704e1de7ae4cf49a3deb166c4e82c9, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.736451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06595ee-4573-4807-b594-8872d24a8543 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.743868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 803/1606] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:06.744506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.759138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 804/1607] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:06 2022] PUT /identity/v3/projects/98c3e8f78b614fa9a55f1a677a5b0d81/users/80704e1de7ae4cf49a3deb166c4e82c9/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:06.763839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d935c1-8b6c-4ecd-8e5d-382e305344ad None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.764556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d935c1-8b6c-4ecd-8e5d-382e305344ad None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.765073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d935c1-8b6c-4ecd-8e5d-382e305344ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.773171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaade4-e75c-4a32-a5c1-bc080781093e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.796864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-76d935c1-8b6c-4ecd-8e5d-382e305344ad None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:06.798200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-76d935c1-8b6c-4ecd-8e5d-382e305344ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:06.867765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9c4fa-b01b-4233-a662-ee42b37b97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.869172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9c4fa-b01b-4233-a662-ee42b37b97fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.869853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9c4fa-b01b-4233-a662-ee42b37b97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.870281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9c4fa-b01b-4233-a662-ee42b37b97fe admin admin] PATH_INFO: `/v3/projects/6a252ae3eeae438cbbc0df47dd50f4e1/users/1d7f41ee02c04671ba1d62f0acd2600e/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.890636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 805/1608] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:06.892003 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23c0d-5383-4cd9-8df9-3f2af3113ae6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.892228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23c0d-5383-4cd9-8df9-3f2af3113ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.892499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23c0d-5383-4cd9-8df9-3f2af3113ae6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.897033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 806/1609] 10.222.0.81 () {54 vars in 962 bytes} [Wed Aug 24 03:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:06.897033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e6ce-435d-41ea-8d50-7c8ab1c4d2ff None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.897033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e6ce-435d-41ea-8d50-7c8ab1c4d2ff None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.897033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e6ce-435d-41ea-8d50-7c8ab1c4d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.905930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9c4fa-b01b-4233-a662-ee42b37b97fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a252ae3eeae438cbbc0df47dd50f4e1, user_id=1d7f41ee02c04671ba1d62f0acd2600e, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.910455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9c4fa-b01b-4233-a662-ee42b37b97fe admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.918348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-60c6e6ce-435d-41ea-8d50-7c8ab1c4d2ff None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:06.921337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-60c6e6ce-435d-41ea-8d50-7c8ab1c4d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:06.938389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 804/1610] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:06 2022] PUT /identity/v3/projects/6a252ae3eeae438cbbc0df47dd50f4e1/users/1d7f41ee02c04671ba1d62f0acd2600e/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:06.938389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d279b-36ff-4cd7-a540-c9a323da6631 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.938389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d279b-36ff-4cd7-a540-c9a323da6631 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.938389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d279b-36ff-4cd7-a540-c9a323da6631 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.938389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 805/1611] 10.222.0.81 () {54 vars in 962 bytes} [Wed Aug 24 03:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:06.938389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d279b-36ff-4cd7-a540-c9a323da6631 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.022346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 807/1612] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:07.026927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155eac1-a061-4f19-824f-6895a431716e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.026927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155eac1-a061-4f19-824f-6895a431716e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.028744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155eac1-a061-4f19-824f-6895a431716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.051235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d279b-36ff-4cd7-a540-c9a323da6631 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.055432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d03ff-f4bd-434a-a985-1c666be9019a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.056095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d03ff-f4bd-434a-a985-1c666be9019a service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.056371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d03ff-f4bd-434a-a985-1c666be9019a service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.056644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d03ff-f4bd-434a-a985-1c666be9019a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.066637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8155eac1-a061-4f19-824f-6895a431716e None None] MFA Rules not processed for user `80704e1de7ae4cf49a3deb166c4e82c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:07.067873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8155eac1-a061-4f19-824f-6895a431716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:07.078275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.128283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d03ff-f4bd-434a-a985-1c666be9019a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.129813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d03ff-f4bd-434a-a985-1c666be9019a service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.135774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.142065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 806/1613] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:07.144305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297dc1-7f14-45e6-b3ea-cd7eaecddc64 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.145417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297dc1-7f14-45e6-b3ea-cd7eaecddc64 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.146027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297dc1-7f14-45e6-b3ea-cd7eaecddc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.157571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 808/1614] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:07.158437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155eac1-a061-4f19-824f-6895a431716e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.200833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ee297dc1-7f14-45e6-b3ea-cd7eaecddc64 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:07.202837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ee297dc1-7f14-45e6-b3ea-cd7eaecddc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:07.230786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155eac1-a061-4f19-824f-6895a431716e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.235437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a6826-183a-4700-a3f9-05ea8788df17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.236056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6826-183a-4700-a3f9-05ea8788df17 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.236337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6826-183a-4700-a3f9-05ea8788df17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.236607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6826-183a-4700-a3f9-05ea8788df17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.248665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6826-183a-4700-a3f9-05ea8788df17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.250184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6826-183a-4700-a3f9-05ea8788df17 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.262621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 809/1615] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:07.264147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.264655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.265090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.295249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 807/1616] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:07.297977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.298321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.298595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.316504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] MFA Rules not processed for user `1d7f41ee02c04671ba1d62f0acd2600e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:07.317962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:07.348043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:07.350482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:07.365202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 808/1617] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:07.366104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.377833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706118c-88a3-48b9-af78-c6ece0b8dba0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.384418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.385204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.385535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.385946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.399387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.400304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.413265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 809/1618] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:07.414944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.427129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34183-8c92-4046-970c-ed0baed1ff67 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.434054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.434991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.434991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.435398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.438985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 810/1619] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:07.439211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.439306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.439407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.465370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acd1f7-6418-42cb-bde0-836abb5407a6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.477959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.478184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 810/1620] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:07.478315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.478315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.478602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.479040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.550268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.608721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.610761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.637763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 811/1621] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:07.638874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.645865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33471b8-534d-426b-a1ce-fcceb4b67fd5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.654538 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.655280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.655561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.655830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.658323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82373-181b-494e-99e5-8e9109bf3875 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.713584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.770250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.776046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.796512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.797460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.797851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.798369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.799109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.803838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.826676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 811/1622] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:07.830095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.862062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 812/1623] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:07.865783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.867884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cba84-7723-409a-a05c-bcda810dbe62 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.015321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.016232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.016654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.017419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.018576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.021979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.029155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be985685-9fdb-46ea-8e36-cbcbbdab796b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.045410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.046445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.047422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.048157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.049057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.051886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.082106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 812/1624] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:08.084504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.113097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 813/1625] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:30:08.113336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d3bc-4560-40f1-9fd5-77deafea2485 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.114487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.129725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.133897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.134663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.134952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.138730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.143808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.154244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77df673-1329-4d86-bb72-d7891d749dcf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.178978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.180239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 813/1626] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:08.180754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.181195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.181626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.181825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.183014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.187343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.208068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01352ad7-c494-4954-b6cc-c9ba79432a27 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.223666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.224681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.225305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.225754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] PATH_INFO: `/v3/projects/407be9e407ae43a3b216dca529060815/users/f3c3e42251844d298915da8a78c8c966/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.241878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 814/1627] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:08.242908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.262659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab0749-7a88-47a8-9601-49dbdf4e5cb6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.277839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407be9e407ae43a3b216dca529060815, user_id=f3c3e42251844d298915da8a78c8c966, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.283773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.284451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.284722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.285019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] PATH_INFO: `/v3/projects/24ce709f07234169802b6bcef3cde279/users/e4864f467ab14350b25fb2b4bfb31764/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.286202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.314696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 814/1628] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:08 2022] PUT /identity/v3/projects/407be9e407ae43a3b216dca529060815/users/f3c3e42251844d298915da8a78c8c966/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:08.322218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.341305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ce709f07234169802b6bcef3cde279, user_id=e4864f467ab14350b25fb2b4bfb31764, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.347414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.391206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 815/1629] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:08 2022] PUT /identity/v3/projects/24ce709f07234169802b6bcef3cde279/users/e4864f467ab14350b25fb2b4bfb31764/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:08.392105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.462457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.523173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46974474-03f7-47a0-a869-35fd8195a052 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.629033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb5c64-d043-43be-bb6d-ac8e76c407d1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.653393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d078bff-55dc-4cba-a07f-823abe40dc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.654268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d078bff-55dc-4cba-a07f-823abe40dc77 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.654727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d078bff-55dc-4cba-a07f-823abe40dc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.655185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d078bff-55dc-4cba-a07f-823abe40dc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.656025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d078bff-55dc-4cba-a07f-823abe40dc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.658218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d078bff-55dc-4cba-a07f-823abe40dc77 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.689513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.723209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 816/1630] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:08.726845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d0e81-e870-43ce-b264-11a83d7d9d84 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.727222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d0e81-e870-43ce-b264-11a83d7d9d84 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.727528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d0e81-e870-43ce-b264-11a83d7d9d84 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.738850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 817/1631] 10.222.0.81 () {54 vars in 912 bytes} [Wed Aug 24 03:30:08 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:08.739718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d0e81-e870-43ce-b264-11a83d7d9d84 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.766426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df6a0-765a-41d9-a9de-0b3bcc1d104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.767249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df6a0-765a-41d9-a9de-0b3bcc1d104e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.767529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df6a0-765a-41d9-a9de-0b3bcc1d104e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.767798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df6a0-765a-41d9-a9de-0b3bcc1d104e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.768864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df6a0-765a-41d9-a9de-0b3bcc1d104e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.770257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df6a0-765a-41d9-a9de-0b3bcc1d104e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.775822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d0e81-e870-43ce-b264-11a83d7d9d84 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.797706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.798358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.799021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.799304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] PATH_INFO: `/v3/projects/24ce709f07234169802b6bcef3cde279/users/e4864f467ab14350b25fb2b4bfb31764/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.799761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 815/1632] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:08.802225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ce709f07234169802b6bcef3cde279, user_id=e4864f467ab14350b25fb2b4bfb31764, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.802371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3d813-a455-4b8b-afd7-1617b6a1a7fd None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.802630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3d813-a455-4b8b-afd7-1617b6a1a7fd None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.802908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3d813-a455-4b8b-afd7-1617b6a1a7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.812729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.841354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 818/1633] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:08 2022] PUT /identity/v3/projects/24ce709f07234169802b6bcef3cde279/users/e4864f467ab14350b25fb2b4bfb31764/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:08.842295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.868168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c3f3d813-a455-4b8b-afd7-1617b6a1a7fd None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:08.874193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c3f3d813-a455-4b8b-afd7-1617b6a1a7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:08.931252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8de52d-75a8-4af1-a53c-b8d6df831312 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.937796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0040827-3053-4295-91b5-9d6cd6e1468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.938591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0040827-3053-4295-91b5-9d6cd6e1468a admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.938794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0040827-3053-4295-91b5-9d6cd6e1468a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.939068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0040827-3053-4295-91b5-9d6cd6e1468a admin admin] PATH_INFO: `/v3/projects/407be9e407ae43a3b216dca529060815/users/f3c3e42251844d298915da8a78c8c966/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.941227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0040827-3053-4295-91b5-9d6cd6e1468a admin admin] RBAC: Authorizing `identity:create_grant(project_id=407be9e407ae43a3b216dca529060815, user_id=f3c3e42251844d298915da8a78c8c966, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.945294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0040827-3053-4295-91b5-9d6cd6e1468a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.958192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 819/1634] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:08 2022] PUT /identity/v3/projects/407be9e407ae43a3b216dca529060815/users/f3c3e42251844d298915da8a78c8c966/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:08.959708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5df1e-e2f1-43c6-ada6-9bd44937dae0 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.959846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5df1e-e2f1-43c6-ada6-9bd44937dae0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.960109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5df1e-e2f1-43c6-ada6-9bd44937dae0 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.970307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 820/1635] 10.222.0.81 () {54 vars in 912 bytes} [Wed Aug 24 03:30:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:08.971785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d652-0167-45f9-be92-fd09808b2dc3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.972112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d652-0167-45f9-be92-fd09808b2dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.972395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d652-0167-45f9-be92-fd09808b2dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.006426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c3a9d652-0167-45f9-be92-fd09808b2dc3 None None] MFA Rules not processed for user `e4864f467ab14350b25fb2b4bfb31764`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:09.008385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c3a9d652-0167-45f9-be92-fd09808b2dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:09.060187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 816/1636] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:30:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:09.067017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.067419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.067793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.095957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 821/1637] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:09.099998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.100281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.103651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.130828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] MFA Rules not processed for user `f3c3e42251844d298915da8a78c8c966`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:09.134023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:09.161901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:09.163142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:09.206473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:09.347355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 817/1638] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:09.349201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.386694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 822/1639] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:30:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:09.389853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.464628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:09.470127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780cf2-e344-4cdd-9260-4238b55c5fb5 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.474698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.475910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.476284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.476971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.490520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.492947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.508709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 818/1640] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:09.561779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfa61e-dabe-4e3f-a7d5-dc4376c6a2fa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.574008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.575614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.575992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.576311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.603096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.605656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.621827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 823/1641] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:09.763211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.776490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e3cb1-98bb-45bf-9fb8-f5bec9036458 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.786515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.788077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.790204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.790985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.817174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.822168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.851680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 819/1642] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:10.018007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:10.034837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98fb7-dac6-47d6-8a7d-316b286168d5 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:10.052465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:10.053492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:10.054115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:10.055127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:10.081895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:10.088002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:10.146172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 824/1643] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:10.353064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:10.362595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e62b53-1195-4c87-a19d-8604a0184038 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:10.376356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aba313-cfff-498c-a64e-961f4884315c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:10.377752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aba313-cfff-498c-a64e-961f4884315c service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:10.379268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aba313-cfff-498c-a64e-961f4884315c service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:10.380215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aba313-cfff-498c-a64e-961f4884315c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:10.549035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aba313-cfff-498c-a64e-961f4884315c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:10.560153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aba313-cfff-498c-a64e-961f4884315c service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:10.593081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 820/1644] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:11.103203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:11.304323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474900e0-0b8f-4b69-b554-73e5439358e2 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:11.322937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:11.324451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.324727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.325157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] PATH_INFO: `/v3/users/e3f543c8ec9f4cdcb7d1dfa0b37c094f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.333983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f543c8ec9f4cdcb7d1dfa0b37c094f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:11.339452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:11.339452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Aug 24 03:30:11.339452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: Aug 24 03:30:11.343024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:11.528784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:11.534142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f543c8ec9f4cdcb7d1dfa0b37c094f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:11.552627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f543c8ec9f4cdcb7d1dfa0b37c094f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:11.622635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f543c8ec9f4cdcb7d1dfa0b37c094f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:11.623639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f543c8ec9f4cdcb7d1dfa0b37c094f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:11.673361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f543c8ec9f4cdcb7d1dfa0b37c094f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:11.701640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 825/1645] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:11 2022] DELETE /identity/v3/users/e3f543c8ec9f4cdcb7d1dfa0b37c094f => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:11.706243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b4b3d-238d-4bce-be37-7493f260d619 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.707231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b4b3d-238d-4bce-be37-7493f260d619 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.707766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b4b3d-238d-4bce-be37-7493f260d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.768298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-710b4b3d-238d-4bce-be37-7493f260d619 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:11.783547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-710b4b3d-238d-4bce-be37-7493f260d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:12.108581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 821/1646] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:12.145808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.147863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:12.387073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c7fca-a3a4-4fd1-a628-6ccdb22102d9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:12.397769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa56433-54dd-4556-b2d1-08077f8d4d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:12.401514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa56433-54dd-4556-b2d1-08077f8d4d33 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.401514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa56433-54dd-4556-b2d1-08077f8d4d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.401840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa56433-54dd-4556-b2d1-08077f8d4d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.438777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.439095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.439386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.443351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa56433-54dd-4556-b2d1-08077f8d4d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:12.449484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa56433-54dd-4556-b2d1-08077f8d4d33 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:12.477813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 826/1647] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:12.526620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] MFA Rules not processed for user `33b932f3feb342d889155c3c34407a88`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:12.527872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:12.675564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 822/1648] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:12.687021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.687308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.687591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.737546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] MFA Rules not processed for user `33b932f3feb342d889155c3c34407a88`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:12.745626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:12.774588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 827/1649] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:12.785631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.789161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:12.849103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.126921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8ceec-ba14-467b-b9c5-07d848a38e6b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.134144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6d8f-5e8b-4eb7-bdb1-247da2890a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.134876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6d8f-5e8b-4eb7-bdb1-247da2890a03 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.135156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6d8f-5e8b-4eb7-bdb1-247da2890a03 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.135861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6d8f-5e8b-4eb7-bdb1-247da2890a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.146094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d6d8f-5e8b-4eb7-bdb1-247da2890a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.150914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d6d8f-5e8b-4eb7-bdb1-247da2890a03 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.173613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 823/1650] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:13.252436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:13.255908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:13.277562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ece4d0-627b-45b5-8da7-bb66e8d4f232 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.290632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.292577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.293091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.299235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] PATH_INFO: `/v3/users/80704e1de7ae4cf49a3deb166c4e82c9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.301622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=80704e1de7ae4cf49a3deb166c4e82c9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.306024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.480090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80704e1de7ae4cf49a3deb166c4e82c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:13.512900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80704e1de7ae4cf49a3deb166c4e82c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:13.561220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80704e1de7ae4cf49a3deb166c4e82c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:13.563291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80704e1de7ae4cf49a3deb166c4e82c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:13.575702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80704e1de7ae4cf49a3deb166c4e82c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:13.590570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 828/1651] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:13 2022] DELETE /identity/v3/users/80704e1de7ae4cf49a3deb166c4e82c9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:13.591860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.592893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.593330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.645129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:13.655563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:13.918380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 824/1652] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:14.006891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.009310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.101339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.173165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04677a8-839e-49aa-b3e6-96f7fdf115ec admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.182155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028d62c-5049-4fa7-a50b-6254410bf93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.184263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028d62c-5049-4fa7-a50b-6254410bf93a service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.186011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028d62c-5049-4fa7-a50b-6254410bf93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.186573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028d62c-5049-4fa7-a50b-6254410bf93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.206179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028d62c-5049-4fa7-a50b-6254410bf93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.208330 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028d62c-5049-4fa7-a50b-6254410bf93a service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.222721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 829/1653] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:14.289515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.291454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.308398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a3199-44e3-4ef3-9fd6-1b385a19d7d4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.326275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.327712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.328336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.328850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] PATH_INFO: `/v3/projects/6f655a873ed64b69ac5108f587f8e040` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.330457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f655a873ed64b69ac5108f587f8e040)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.332183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: WARNING py.warnings [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:14.332183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: warnings.warn(msg) Aug 24 03:30:14.332183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: Aug 24 03:30:14.334568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.410723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f655a873ed64b69ac5108f587f8e040', 'request_id': 'req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:14.425907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f655a873ed64b69ac5108f587f8e040', 'request_id': 'req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:14.435180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f655a873ed64b69ac5108f587f8e040', 'request_id': 'req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:14.450523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] The token cache is being invalidate because project 6f655a873ed64b69ac5108f587f8e040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:14.451038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:14.452755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bc28e24b-456b-4884-a513-ed9a8d3f6f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:14.454430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 825/1654] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:14 2022] DELETE /identity/v3/projects/6f655a873ed64b69ac5108f587f8e040 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:14.461491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb5f49-8217-4a04-9d79-2998f2e8df63 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.462418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb5f49-8217-4a04-9d79-2998f2e8df63 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.462418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb5f49-8217-4a04-9d79-2998f2e8df63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.501901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-98fb5f49-8217-4a04-9d79-2998f2e8df63 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:14.503666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-98fb5f49-8217-4a04-9d79-2998f2e8df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.546724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab35624-1688-4d8d-9106-149e6597b5d9 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.547072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab35624-1688-4d8d-9106-149e6597b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.547490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab35624-1688-4d8d-9106-149e6597b5d9 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.549346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 826/1655] 10.222.0.81 () {54 vars in 912 bytes} [Wed Aug 24 03:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:14.563384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.563681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.563958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.688599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.704239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:14.707873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.755018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 830/1656] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:14.773730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.778779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.779652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.836833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:14.836833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.842618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 827/1657] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:30:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:14.850928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 831/1658] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:14.855629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.872162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe6019-a344-466b-9a94-ed6265f927f4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.878702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.879637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.880554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.880885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.881564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:14.881872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.884328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.888909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.891248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.912734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87f3e2-5fae-40a5-938f-1615e822cae9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.920235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.922269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.922865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.923142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.930909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 828/1659] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:14.935015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.964727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.967617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.977349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2ca5-6563-45c1-ab15-6dac49a99843 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.991271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 832/1660] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:14.991924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.993540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.993819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.994707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.995375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.998414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.061905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 829/1661] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:14 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:15.065776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.083833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301b92-e3da-481f-b0f6-0cd1190dd102 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.231685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.232374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.232662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.232942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.233625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.235548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.294944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 833/1662] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:15 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:15.299401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.324799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b828-f9c9-4b0b-ba8d-686718b95873 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.330470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.331248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.331588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.331872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.332531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.334745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.364718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 830/1663] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:15.369258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.386796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa362-a984-4f7b-b75a-52dc6dadae60 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.403538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490b2ed-0969-401e-9541-04021931e7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.404434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490b2ed-0969-401e-9541-04021931e7eb admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.404797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490b2ed-0969-401e-9541-04021931e7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.405158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490b2ed-0969-401e-9541-04021931e7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.405900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490b2ed-0969-401e-9541-04021931e7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.407864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490b2ed-0969-401e-9541-04021931e7eb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.450110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 834/1664] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:15.454797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.471763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3261cb-2e2c-411d-bead-1128f06dc161 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.483969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.484657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.484965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.485530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] PATH_INFO: `/v3/projects/abaa6b53cb4145a697de678d0debcc41/users/27ff216440a948d5b3a97bd942eb7398/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.525818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abaa6b53cb4145a697de678d0debcc41, user_id=27ff216440a948d5b3a97bd942eb7398, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.528856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.588595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 831/1665] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:15 2022] PUT /identity/v3/projects/abaa6b53cb4145a697de678d0debcc41/users/27ff216440a948d5b3a97bd942eb7398/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:15.600856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.601348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.604648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.607865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.611534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:15.709192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] MFA Rules not processed for user `27ff216440a948d5b3a97bd942eb7398`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:15.710446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:15.769588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.828049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67156495-dde0-477c-9eff-58554406d6d9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.850853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 835/1666] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:15.852241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.853274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.855774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.856090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.856442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] PATH_INFO: `/v3/projects/98c3e8f78b614fa9a55f1a677a5b0d81` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.860429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c3e8f78b614fa9a55f1a677a5b0d81)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.864394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.871282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891374e1-7b72-45f9-90a3-402f4dd7edf8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.879048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.879730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.880001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.880320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.881164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.884671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.929290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 836/1667] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:15 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:15.951610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.055467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.064742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c3e8f78b614fa9a55f1a677a5b0d81', 'request_id': 'req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.082980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c3e8f78b614fa9a55f1a677a5b0d81', 'request_id': 'req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.099282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c3e8f78b614fa9a55f1a677a5b0d81', 'request_id': 'req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.123843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] The token cache is being invalidate because project 98c3e8f78b614fa9a55f1a677a5b0d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:16.124169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.126132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e00dee-ce35-4e2f-abe3-b1d9bf60c808 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.126910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.135810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 832/1668] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:15 2022] DELETE /identity/v3/projects/98c3e8f78b614fa9a55f1a677a5b0d81 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:16.136958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.145345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.301723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.302311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.302595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.302870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.308390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.312218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.324422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12203c-ec8b-4066-8cdb-5a25ba9f0c2c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.344360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.344961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.345252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.345521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] PATH_INFO: `/v3/users/f3c3e42251844d298915da8a78c8c966` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.346697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c3e42251844d298915da8a78c8c966)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.348744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.388541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 837/1669] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:15 2022] POST /identity/v3/users => generated 295 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:16.393641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.399101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.474477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bffcd3-668a-456d-81e6-75891e5c7a9d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.502298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c3e42251844d298915da8a78c8c966'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.511345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fca16-3dac-4f7c-9372-0057299a6eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.512016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fca16-3dac-4f7c-9372-0057299a6eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.512303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fca16-3dac-4f7c-9372-0057299a6eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.512634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fca16-3dac-4f7c-9372-0057299a6eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.513493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fca16-3dac-4f7c-9372-0057299a6eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.515218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fca16-3dac-4f7c-9372-0057299a6eb6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.554528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c3e42251844d298915da8a78c8c966'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.556145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 838/1670] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:16.557408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208bca6-2411-49c9-9888-ac104c2fd0d0 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.557694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208bca6-2411-49c9-9888-ac104c2fd0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.557972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208bca6-2411-49c9-9888-ac104c2fd0d0 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.559421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c3e42251844d298915da8a78c8c966'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.570791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 839/1671] 10.222.0.81 () {54 vars in 961 bytes} [Wed Aug 24 03:30:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:16.571705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208bca6-2411-49c9-9888-ac104c2fd0d0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.581167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1208bca6-2411-49c9-9888-ac104c2fd0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.582302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c3e42251844d298915da8a78c8c966'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.637383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-deadfa36-0a90-4bb7-86b5-03caa002fba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c3e42251844d298915da8a78c8c966'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:16.654123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208bca6-2411-49c9-9888-ac104c2fd0d0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.658477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 833/1672] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:16 2022] DELETE /identity/v3/users/f3c3e42251844d298915da8a78c8c966 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:16.660352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.660643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.660949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.666588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9634baf-b9c4-4763-9e2b-ec49f6a85a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.667309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9634baf-b9c4-4763-9e2b-ec49f6a85a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.674849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9634baf-b9c4-4763-9e2b-ec49f6a85a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.675214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9634baf-b9c4-4763-9e2b-ec49f6a85a22 admin admin] PATH_INFO: `/v3/projects/ecefc97011d847279a0795a6bed21e83/users/7cf1dca5e27f4136bb3a74a1ba070371/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.695326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9634baf-b9c4-4763-9e2b-ec49f6a85a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecefc97011d847279a0795a6bed21e83, user_id=7cf1dca5e27f4136bb3a74a1ba070371, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.698921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9634baf-b9c4-4763-9e2b-ec49f6a85a22 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.736907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 840/1673] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:16 2022] PUT /identity/v3/projects/ecefc97011d847279a0795a6bed21e83/users/7cf1dca5e27f4136bb3a74a1ba070371/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:16.739423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.739791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.741668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.744313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:16.747075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.753447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.771422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] MFA Rules not processed for user `33b932f3feb342d889155c3c34407a88`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:16.773238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.866129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.905116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 841/1674] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:16.905116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.942424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 834/1675] 10.222.0.81 () {58 vars in 1058 bytes} [Wed Aug 24 03:30:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:16.953993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.977262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd56648-f0b0-4007-b6cd-38346b2c09bd None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.003325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.003948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.004225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.005064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.016655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:17.037131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.040051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.066796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 835/1676] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:17.112547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a17558-5e25-4d3a-88d1-7970dca90f2b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.132429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9159e2-c75c-4bee-a000-8529237f1ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.134202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9159e2-c75c-4bee-a000-8529237f1ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.134648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9159e2-c75c-4bee-a000-8529237f1ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.134927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9159e2-c75c-4bee-a000-8529237f1ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.135596 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9159e2-c75c-4bee-a000-8529237f1ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.145629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9159e2-c75c-4bee-a000-8529237f1ce1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.223742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 842/1677] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:17.229838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.250688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0f14-95f6-4ebb-b9fc-700ff2129f79 service placement] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.265151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.267406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.268398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.269070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] PATH_INFO: `/v3/projects/ecefc97011d847279a0795a6bed21e83/users/7cf1dca5e27f4136bb3a74a1ba070371/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.274301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecefc97011d847279a0795a6bed21e83, user_id=7cf1dca5e27f4136bb3a74a1ba070371, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.284397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.340127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 836/1678] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:17 2022] PUT /identity/v3/projects/ecefc97011d847279a0795a6bed21e83/users/7cf1dca5e27f4136bb3a74a1ba070371/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:17.345980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ff14-d13c-429f-a39f-a7f324c17ad8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.346472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ff14-d13c-429f-a39f-a7f324c17ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.346892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75ff14-d13c-429f-a39f-a7f324c17ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.439248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8c75ff14-d13c-429f-a39f-a7f324c17ad8 None None] MFA Rules not processed for user `7cf1dca5e27f4136bb3a74a1ba070371`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:17.441815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8c75ff14-d13c-429f-a39f-a7f324c17ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:17.802124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 843/1679] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:18.015717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:18.017985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:18.245648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4539a-85e6-44dd-ada3-7de4310d9398 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.245930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4539a-85e6-44dd-ada3-7de4310d9398 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.246233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4539a-85e6-44dd-ada3-7de4310d9398 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.248190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 844/1680] 10.222.0.81 () {54 vars in 912 bytes} [Wed Aug 24 03:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:18.253970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.254217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.254416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.280164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ab62f-39ba-4699-a5f5-ca4b82538485 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:18.296678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:18.298194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.299930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.300778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.309625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:18.321407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:18.325489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:18.537958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:18.654886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:18.666761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:18.689153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 845/1681] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:30:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:18.690098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:18.701033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 837/1682] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:18.701988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:18.706211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:18.708226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:18.746488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59326b-ff30-47b4-a89f-fd3f282e8719 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:18.751866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22453-a054-4aa6-9353-a103c3ff606b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:18.753138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22453-a054-4aa6-9353-a103c3ff606b service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.753441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22453-a054-4aa6-9353-a103c3ff606b service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.753718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22453-a054-4aa6-9353-a103c3ff606b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.763301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22453-a054-4aa6-9353-a103c3ff606b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:18.765027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22453-a054-4aa6-9353-a103c3ff606b service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:18.776370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 838/1683] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:19.028059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d06f19-f17b-47bd-8713-ea415b67a7a4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:19.049937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:19.050602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.054382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.055409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] PATH_INFO: `/v3/projects/407be9e407ae43a3b216dca529060815` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.059064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=407be9e407ae43a3b216dca529060815)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:19.066253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:19.233100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407be9e407ae43a3b216dca529060815', 'request_id': 'req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:19.241431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407be9e407ae43a3b216dca529060815', 'request_id': 'req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:19.252123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407be9e407ae43a3b216dca529060815', 'request_id': 'req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:19.291227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] The token cache is being invalidate because project 407be9e407ae43a3b216dca529060815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:19.294004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:19.294984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b0b2e23-2ccc-46db-895c-90c40c7bfa6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:19.298782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 846/1684] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:18 2022] DELETE /identity/v3/projects/407be9e407ae43a3b216dca529060815 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:19.304247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.305496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.306434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.350981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:19.356354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:19.585474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 839/1685] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:19.587638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.587932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.588215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.629445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:19.632444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:19.663787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 847/1686] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:19.667638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.674697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:19.683948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cd39a-1f8a-4e77-af61-a3282616bb58 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:19.707795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:19.708607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.710920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.712048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.714231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:19.715676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:19.729962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:19.752955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.761458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 840/1687] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:19.766451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50e14e-13d2-4a0b-b309-eb4461497571 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:19.786897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:19.787534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.787812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.788088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.788840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:19.790791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:19.855084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.865774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 848/1688] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:30:19.868696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e47905-7015-42a4-a789-52ea1f62dd92 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:19.970693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:19.971416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.971774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.972122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.972933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:19.975018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.023353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.038174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071ad50-046b-4b4b-adaa-66b8be6fd722 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.039199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 841/1689] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/users => generated 282 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:20.049633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.056472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.056738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.058032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.058852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.060411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.124909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.125808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 849/1690] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:30:20.148661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98338f8c-b658-43cc-b27d-d26fb58ef4b8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.167807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69195e4-5ea3-4ec7-8e92-ef5733a48fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.168796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69195e4-5ea3-4ec7-8e92-ef5733a48fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.169003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69195e4-5ea3-4ec7-8e92-ef5733a48fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.169352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69195e4-5ea3-4ec7-8e92-ef5733a48fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.170482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69195e4-5ea3-4ec7-8e92-ef5733a48fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.172142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69195e4-5ea3-4ec7-8e92-ef5733a48fdf admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.216059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 842/1691] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:20.216415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.245742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bf3b6-3829-43cb-80eb-b85a2e75e4bb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.260051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.260739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.261355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.261642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] PATH_INFO: `/v3/projects/e35cb6778b4046e79d1d288f9585d228/users/97be82768e404a6ab13398a24a710912/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.300935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35cb6778b4046e79d1d288f9585d228, user_id=97be82768e404a6ab13398a24a710912, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.304620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.341940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 850/1692] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:20 2022] PUT /identity/v3/projects/e35cb6778b4046e79d1d288f9585d228/users/97be82768e404a6ab13398a24a710912/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:20.342394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.342867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.342984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.383909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] MFA Rules not processed for user `97be82768e404a6ab13398a24a710912`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:20.386209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:20.515758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 843/1693] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:20.524441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.542712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.775184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e1218-7343-49a8-a114-b5192c588c22 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.787496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.789619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.790709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.790998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.792730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.798146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.884579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.884907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 851/1694] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:20 2022] POST /identity/v3/projects => generated 363 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:20.926927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89001a92-8727-4804-b16d-57a11620f5f7 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.007122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.018536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.019164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.019445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.019719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.020357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.022585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.061698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.062177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 844/1695] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:20 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:21.096870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cb167-1630-460c-b5a2-3b9e931d3701 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.120522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.123081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.125625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.125978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.126711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.128375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.179616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.184718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 852/1696] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:21.205650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bce731-613e-4b87-96b9-109e77472dba admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.222530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85447f0-956d-4d4c-9bb3-05c40b72def4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.224449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85447f0-956d-4d4c-9bb3-05c40b72def4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.225646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85447f0-956d-4d4c-9bb3-05c40b72def4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.227744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85447f0-956d-4d4c-9bb3-05c40b72def4 admin admin] PATH_INFO: `/v3/projects/bfbd362ed86c407e92d4b2ecf55c45d4/users/5822e1c54f514dc490114948f4ca3d83/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.283861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85447f0-956d-4d4c-9bb3-05c40b72def4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbd362ed86c407e92d4b2ecf55c45d4, user_id=5822e1c54f514dc490114948f4ca3d83, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.290008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85447f0-956d-4d4c-9bb3-05c40b72def4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.324645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 845/1697] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:21 2022] PUT /identity/v3/projects/bfbd362ed86c407e92d4b2ecf55c45d4/users/5822e1c54f514dc490114948f4ca3d83/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:21.327162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.345802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb918afe-46c2-4913-99e1-ba6995340607 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.467139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f98cf-8555-4035-8966-6c7ed209eab2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.468204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f98cf-8555-4035-8966-6c7ed209eab2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.468582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f98cf-8555-4035-8966-6c7ed209eab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.498117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.498740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.499014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.499285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.499934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.509337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d03f98cf-8555-4035-8966-6c7ed209eab2 None None] MFA Rules not processed for user `ce4bf5ffbf2d47c9b54c4c1e343e8944`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:21.510970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d03f98cf-8555-4035-8966-6c7ed209eab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.512087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.538185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 853/1698] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:21.543389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.572132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455da6b-8938-48cb-904d-c83371b23328 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.588034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe282a-dc11-48cc-a2be-046f54a2fbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.588725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe282a-dc11-48cc-a2be-046f54a2fbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.589045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe282a-dc11-48cc-a2be-046f54a2fbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.589318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe282a-dc11-48cc-a2be-046f54a2fbcf admin admin] PATH_INFO: `/v3/projects/bfbd362ed86c407e92d4b2ecf55c45d4/users/5822e1c54f514dc490114948f4ca3d83/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.595510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe282a-dc11-48cc-a2be-046f54a2fbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbd362ed86c407e92d4b2ecf55c45d4, user_id=5822e1c54f514dc490114948f4ca3d83, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.598652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe282a-dc11-48cc-a2be-046f54a2fbcf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.627472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 854/1699] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:21 2022] PUT /identity/v3/projects/bfbd362ed86c407e92d4b2ecf55c45d4/users/5822e1c54f514dc490114948f4ca3d83/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:21.638884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.639206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.639413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.685025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] MFA Rules not processed for user `5822e1c54f514dc490114948f4ca3d83`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:21.690132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.714578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 846/1700] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:21.719161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.719516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.719737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.753998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] MFA Rules not processed for user `ce4bf5ffbf2d47c9b54c4c1e343e8944`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:21.756128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.772996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.815690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 855/1701] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:21.834899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.837960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.888300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 847/1702] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:21.900235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.901939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.965949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c15d1d-3fa3-4f6b-a46d-aed2df3f183d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.973878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.975531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.975842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.976209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.999280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.000740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.081784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 856/1703] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:22.086655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:22.125404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd98b2-d45c-480c-a799-51fe1e403c7b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.145266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.145984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.146541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.148392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.178204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.180967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.219233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 848/1704] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:22.269941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.269941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:22.289804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.303507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecd5dd-b88a-4ff1-a1a7-52aa86d97049 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.311376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b33fee-444d-4782-8858-15039a583f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.316642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b33fee-444d-4782-8858-15039a583f58 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.317074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b33fee-444d-4782-8858-15039a583f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.319175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b33fee-444d-4782-8858-15039a583f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.487381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b33fee-444d-4782-8858-15039a583f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.511538 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b33fee-444d-4782-8858-15039a583f58 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.519362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a5aca-6141-4ce6-aed8-223c5674db16 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.542575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.543956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.545708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.547226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] PATH_INFO: `/v3/users/e09d29c12c6e4dd2a0e089f18252cda5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.552446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09d29c12c6e4dd2a0e089f18252cda5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.557391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.567535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 849/1705] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:22.773251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09d29c12c6e4dd2a0e089f18252cda5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:22.795372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09d29c12c6e4dd2a0e089f18252cda5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:22.819701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09d29c12c6e4dd2a0e089f18252cda5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:22.820541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09d29c12c6e4dd2a0e089f18252cda5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:22.832016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09d29c12c6e4dd2a0e089f18252cda5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:22.844713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 857/1706] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:22 2022] DELETE /identity/v3/users/e09d29c12c6e4dd2a0e089f18252cda5 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:22.846432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d098d-3903-4d98-a710-8fd7d35f6755 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.846714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d098d-3903-4d98-a710-8fd7d35f6755 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.846986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d098d-3903-4d98-a710-8fd7d35f6755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.965259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d07d098d-3903-4d98-a710-8fd7d35f6755 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:22.978277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d07d098d-3903-4d98-a710-8fd7d35f6755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.279231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 850/1707] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:23.422466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.423991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.443969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.612006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab1876-868a-4676-8def-45df4c391c4d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.635632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd70ca9-65d7-45c8-b1a1-ae4f9dda8c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.638763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd70ca9-65d7-45c8-b1a1-ae4f9dda8c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.639361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd70ca9-65d7-45c8-b1a1-ae4f9dda8c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.640056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd70ca9-65d7-45c8-b1a1-ae4f9dda8c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.687236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.689033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.689587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.691270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd70ca9-65d7-45c8-b1a1-ae4f9dda8c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:23.692889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd70ca9-65d7-45c8-b1a1-ae4f9dda8c2f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:23.752063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 858/1708] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:23.786700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] MFA Rules not processed for user `66900b813b934514a7920ba975a05ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:23.788745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.855378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.855662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.856004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.982843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:23.985217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:24.029766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 851/1709] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:24.208519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.210762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:24.244300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.247913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 859/1710] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:30:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:24.372490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a393-1b00-4b05-976b-fe1750347065 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.390354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.390885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.391161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.391418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.415548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.417234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.446567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 852/1711] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:24.549561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.552307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:24.769477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82091771-c0c7-40fb-a2b0-5e988bea1569 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.796028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecafe4-108f-4da3-89ed-a56f90b608bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.798010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecafe4-108f-4da3-89ed-a56f90b608bd service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.800082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecafe4-108f-4da3-89ed-a56f90b608bd service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.800537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecafe4-108f-4da3-89ed-a56f90b608bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.812527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.826237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecafe4-108f-4da3-89ed-a56f90b608bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.828395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecafe4-108f-4da3-89ed-a56f90b608bd service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.848655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 860/1712] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:30:25.144542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.147019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:25.192157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b4b1-3db7-4abd-8a59-0903a0010e27 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.213604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.214473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.216692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.217285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] PATH_INFO: `/v3/projects/84fe38d4a2f24b5ba577c169f125bf67` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.221868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=84fe38d4a2f24b5ba577c169f125bf67)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.233640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.458533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84fe38d4a2f24b5ba577c169f125bf67', 'request_id': 'req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:25.482820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84fe38d4a2f24b5ba577c169f125bf67', 'request_id': 'req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:25.501728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84fe38d4a2f24b5ba577c169f125bf67', 'request_id': 'req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:25.531417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] The token cache is being invalidate because project 84fe38d4a2f24b5ba577c169f125bf67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:25.531712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:25.532429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ffa5fd51-3f0c-4bf4-ba15-c40f90a81a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:25.534505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 853/1713] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:25 2022] DELETE /identity/v3/projects/84fe38d4a2f24b5ba577c169f125bf67 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:25.539055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.540575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.540575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.595898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:25.598648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:25.784290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 861/1714] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:25.792744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.793452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.793870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.831751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:25.839438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:25.856821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 854/1715] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:25.857797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.873518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc927d8-671c-4528-8627-e0ecfccec3a9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.891918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.892552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.892981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.893207 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.893873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:25.894171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.895853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.932137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 862/1716] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:25.933566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.947234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcfaaf-99e5-47ae-a625-b0dec9ae05c1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.961518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.962867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.963151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.967317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.968726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.976834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.054383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 855/1717] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:25 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:26.058272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.074623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2c402-8acd-4797-b7d3-d8b12e5bd8d0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.258497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.259173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.259514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.259849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.260565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.264496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.345444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.353593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 863/1718] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:26 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:26.365334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391526a-f897-409c-a125-0ea4d756fb12 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.374056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.374919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.375373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.375838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.376669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.379134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.397421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 856/1719] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:26.402521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.427041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298be61-9750-4569-92e3-25e14b5a08af admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.445645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd94d28-e3c9-4a19-aa56-a96c8e31c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.446587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94d28-e3c9-4a19-aa56-a96c8e31c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.447306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94d28-e3c9-4a19-aa56-a96c8e31c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.447979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94d28-e3c9-4a19-aa56-a96c8e31c0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.449094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd94d28-e3c9-4a19-aa56-a96c8e31c0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.454432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd94d28-e3c9-4a19-aa56-a96c8e31c0ce admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.497290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 864/1720] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:26.501959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.522135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ec05d-af0e-4e5f-99e2-b6d3c54d9f3e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.545975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f48e08-4082-4b14-8534-c9d81e7cf279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.547298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f48e08-4082-4b14-8534-c9d81e7cf279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.547670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f48e08-4082-4b14-8534-c9d81e7cf279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.547830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f48e08-4082-4b14-8534-c9d81e7cf279 admin admin] PATH_INFO: `/v3/projects/51edea3eb09c48f2b346ae28d5b4feeb/users/ac0d24dbfe384380871b207c353b9cee/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.582478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f48e08-4082-4b14-8534-c9d81e7cf279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51edea3eb09c48f2b346ae28d5b4feeb, user_id=ac0d24dbfe384380871b207c353b9cee, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.585359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f48e08-4082-4b14-8534-c9d81e7cf279 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.638263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 857/1721] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:26 2022] PUT /identity/v3/projects/51edea3eb09c48f2b346ae28d5b4feeb/users/ac0d24dbfe384380871b207c353b9cee/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:26.652463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.653040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.653040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.741456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] MFA Rules not processed for user `ac0d24dbfe384380871b207c353b9cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:26.742966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:26.848360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.848692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.848935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.886211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:26.890398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:26.893801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:26.940067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:26.963917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 865/1722] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:26.977975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.980884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:27.160784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded477-0596-48fc-bfb2-710d006ff182 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:27.170962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:27.171608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:27.171853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:27.172678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:27.176010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 858/1723] 10.222.0.84 () {58 vars in 1015 bytes} [Wed Aug 24 03:30:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:27.185820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:27.187711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:27.199181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:27.211322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:27.252352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 866/1724] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:27.260461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:27.263964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbffbf3-3e9d-451a-9e5d-eac71429f983 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:27.268170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:27.269958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:27.270702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:27.271232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:27.271671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:27.294725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:27.301214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:27.343308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 859/1725] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:27.513335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c5a8c-8fb0-4273-9db1-44c750908198 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:27.517978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:27.518523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:27.518805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:27.519089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:27.521036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:27.711039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:27.712579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:27.727183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 867/1726] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:28.135459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:28.137161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:28.410714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83e1db-0dad-4da9-8f67-65d5abd5ff09 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:28.444038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:28.444038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.444038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.444038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] PATH_INFO: `/v3/users/052a827bf2ae437b851572340b67c370` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.445804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=052a827bf2ae437b851572340b67c370)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:28.447761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:28.532298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:28.533946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:28.585426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98dd55-e4ce-48f7-9719-f98d959f77a5 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:28.605867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:28.606969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.614163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.614663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052a827bf2ae437b851572340b67c370'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.615128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] PATH_INFO: `/v3/users/97be82768e404a6ab13398a24a710912` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.620851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97be82768e404a6ab13398a24a710912)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:28.624255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:28.650295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052a827bf2ae437b851572340b67c370'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.658501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052a827bf2ae437b851572340b67c370'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.682244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052a827bf2ae437b851572340b67c370'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.722349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:28.726736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-568cdf57-3b74-4543-9f0a-3f173604d07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052a827bf2ae437b851572340b67c370'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.753500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 860/1727] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:28 2022] DELETE /identity/v3/users/052a827bf2ae437b851572340b67c370 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:28.765288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.765637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.765934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.846451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97be82768e404a6ab13398a24a710912'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.853821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:28.859354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:28.864727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97be82768e404a6ab13398a24a710912'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.921982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97be82768e404a6ab13398a24a710912'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.925466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97be82768e404a6ab13398a24a710912'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.950402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c9cac109-f1fc-497c-9b8e-c2e2e2a4a881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97be82768e404a6ab13398a24a710912'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:28.981054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 868/1728] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:28 2022] DELETE /identity/v3/users/97be82768e404a6ab13398a24a710912 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:28.984163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.987105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.987569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.991118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:29.052168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:29.058741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.170140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 861/1729] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:29.171058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.181782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.217688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c62b2-a07c-4f63-9f90-049679d7da96 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.294681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 869/1730] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:29.364208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.365768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.467120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.467860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.468132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.468562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] PATH_INFO: `/v3/users/1d7f41ee02c04671ba1d62f0acd2600e` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.472441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7f41ee02c04671ba1d62f0acd2600e)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.478668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.528564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28fb4-8fd3-48a5-a97c-05c3fee86080 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.533671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.534500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.534923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.535340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.557631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.561362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.586705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 870/1731] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:29.593109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.603145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7f41ee02c04671ba1d62f0acd2600e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:29.614916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05227f8-549a-47b2-abf4-a0033f3a8154 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.624702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7f41ee02c04671ba1d62f0acd2600e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:29.626059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.626687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.626962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.627065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7f41ee02c04671ba1d62f0acd2600e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:29.627232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.634659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.644707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7f41ee02c04671ba1d62f0acd2600e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:29.666926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.670374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3142a4-2c93-4d0c-9179-c7cb9e4ecf85 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.681539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ea7519c8-f67a-466c-ba0f-cfa1f9bd7539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7f41ee02c04671ba1d62f0acd2600e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:29.702943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 871/1732] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:29.727879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 862/1733] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:29 2022] DELETE /identity/v3/users/1d7f41ee02c04671ba1d62f0acd2600e => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:29.729151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.729640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.729737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.798135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:29.804413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.983960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.984284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.984565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.000707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.029157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] MFA Rules not processed for user `66900b813b934514a7920ba975a05ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:30.030383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:30.171669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 872/1734] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:30.259916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.316857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 863/1735] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:30.346707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.348812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:30.728375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680ba8-b5c6-4b49-822b-381bc706d8b0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.737545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.738167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.738483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.738799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.767526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.769322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.811140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 873/1736] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:32.707643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:32.717951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:32.746548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefdaac-40b6-41ce-8a67-bd271e36fdda None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:32.770353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:32.774330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:32.774871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:32.779116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:32.782373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:32.787662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] PATH_INFO: `/v3/projects/e35cb6778b4046e79d1d288f9585d228` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:32.799902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35cb6778b4046e79d1d288f9585d228)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:32.809271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:32.832709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d3f64-0959-4fda-a260-b0c977381402 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:32.870282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:32.895440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:32.896037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:32.898485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] PATH_INFO: `/v3/projects/6a252ae3eeae438cbbc0df47dd50f4e1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:32.906099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a252ae3eeae438cbbc0df47dd50f4e1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:32.911227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:33.068672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a252ae3eeae438cbbc0df47dd50f4e1', 'request_id': 'req-67a3186a-dfe7-4737-850d-62d7d4fef36e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.083098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a252ae3eeae438cbbc0df47dd50f4e1', 'request_id': 'req-67a3186a-dfe7-4737-850d-62d7d4fef36e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.084182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35cb6778b4046e79d1d288f9585d228', 'request_id': 'req-135b2023-b63f-40d7-982b-8b5b4c014fe3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.108885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a252ae3eeae438cbbc0df47dd50f4e1', 'request_id': 'req-67a3186a-dfe7-4737-850d-62d7d4fef36e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.120737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35cb6778b4046e79d1d288f9585d228', 'request_id': 'req-135b2023-b63f-40d7-982b-8b5b4c014fe3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.143336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35cb6778b4046e79d1d288f9585d228', 'request_id': 'req-135b2023-b63f-40d7-982b-8b5b4c014fe3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.144233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] The token cache is being invalidate because project 6a252ae3eeae438cbbc0df47dd50f4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:33.144895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.145926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.147861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 874/1737] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:32 2022] DELETE /identity/v3/projects/6a252ae3eeae438cbbc0df47dd50f4e1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:33.149115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:33.167190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:33.190943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] The token cache is being invalidate because project e35cb6778b4046e79d1d288f9585d228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:33.194271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.201301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.216058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 864/1738] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:32 2022] DELETE /identity/v3/projects/e35cb6778b4046e79d1d288f9585d228 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:33.217123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:33.227061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:33.438592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b2023-b63f-40d7-982b-8b5b4c014fe3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:33.456060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:33.456721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.457034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.457309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] PATH_INFO: `/v3/users/e4864f467ab14350b25fb2b4bfb31764` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.459645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4864f467ab14350b25fb2b4bfb31764)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:33.462848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:33.542920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:33.582707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4864f467ab14350b25fb2b4bfb31764'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.603163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4864f467ab14350b25fb2b4bfb31764'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.605585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4864f467ab14350b25fb2b4bfb31764'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.621552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3186a-dfe7-4737-850d-62d7d4fef36e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:33.636168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4864f467ab14350b25fb2b4bfb31764'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.671674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:33.672344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.672645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.672912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] PATH_INFO: `/v3/projects/cf053bc966194447b1f624d379f1ff72` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.681160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf053bc966194447b1f624d379f1ff72)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:33.685862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:33.704423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4864f467ab14350b25fb2b4bfb31764'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:33.713499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:33.750829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 865/1739] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:33 2022] DELETE /identity/v3/users/e4864f467ab14350b25fb2b4bfb31764 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:33.766887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:33.774446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:34.045381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf053bc966194447b1f624d379f1ff72', 'request_id': 'req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.071199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf053bc966194447b1f624d379f1ff72', 'request_id': 'req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.094961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf053bc966194447b1f624d379f1ff72', 'request_id': 'req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.169322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] The token cache is being invalidate because project cf053bc966194447b1f624d379f1ff72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:34.169802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.170650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.172359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 875/1740] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:33 2022] DELETE /identity/v3/projects/cf053bc966194447b1f624d379f1ff72 => generated 0 bytes in 1024 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:34.177950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.191501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:34.279013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0efbb-b0ed-431a-b160-b166dcd73010 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.593608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90b0ac-a09a-44b7-8e6b-dbf96beaa524 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.605369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.606037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.606322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.606600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] PATH_INFO: `/v3/users/5822e1c54f514dc490114948f4ca3d83` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.611000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5822e1c54f514dc490114948f4ca3d83)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.615706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.615928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.616409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.616693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.617063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] PATH_INFO: `/v3/users/ac0d24dbfe384380871b207c353b9cee` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.625094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0d24dbfe384380871b207c353b9cee)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.632123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.743024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5822e1c54f514dc490114948f4ca3d83'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.753445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5822e1c54f514dc490114948f4ca3d83'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.753929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5822e1c54f514dc490114948f4ca3d83'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.787133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5822e1c54f514dc490114948f4ca3d83'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.842353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0d24dbfe384380871b207c353b9cee'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.865422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-13c5028b-f317-4dbf-8621-554f59957285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5822e1c54f514dc490114948f4ca3d83'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.866530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0d24dbfe384380871b207c353b9cee'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.904139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 866/1741] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:33 2022] DELETE /identity/v3/users/5822e1c54f514dc490114948f4ca3d83 => generated 0 bytes in 1153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:34.907324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.907606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.907884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.921886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0d24dbfe384380871b207c353b9cee'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.922849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0d24dbfe384380871b207c353b9cee'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.941041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9b92b389-6128-40d5-976d-06c44815dab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0d24dbfe384380871b207c353b9cee'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:34.953519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 876/1742] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:34 2022] DELETE /identity/v3/users/ac0d24dbfe384380871b207c353b9cee => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:34.958253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02aa83-253b-4b0b-8c7f-12643f63251b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.958984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02aa83-253b-4b0b-8c7f-12643f63251b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.960569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02aa83-253b-4b0b-8c7f-12643f63251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.974507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.020547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6d02aa83-253b-4b0b-8c7f-12643f63251b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:35.021848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:35.022501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6d02aa83-253b-4b0b-8c7f-12643f63251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:35.034039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:35.194274 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.238415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 877/1743] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:35.383607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 867/1744] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:35.392079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a77d-e40b-4ee9-a7ce-d9cf97a21f31 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.392375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a77d-e40b-4ee9-a7ce-d9cf97a21f31 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.392643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a77d-e40b-4ee9-a7ce-d9cf97a21f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.402411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.405746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:35.436822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-aa13a77d-e40b-4ee9-a7ce-d9cf97a21f31 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:35.439477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aa13a77d-e40b-4ee9-a7ce-d9cf97a21f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:35.496256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 878/1745] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:35.498782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2038dd-6c43-46ff-a26d-31037b968df1 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.512657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2038dd-6c43-46ff-a26d-31037b968df1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.515711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2038dd-6c43-46ff-a26d-31037b968df1 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.522618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 879/1746] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:30:35 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:35.523710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2038dd-6c43-46ff-a26d-31037b968df1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.603967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2038dd-6c43-46ff-a26d-31037b968df1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.631431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.632058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.632338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.636439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.637771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:35.638086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.642099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.696365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 880/1747] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:35.714822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.744572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc965995-4af7-444d-ad25-69b55a9f8df0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.755502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.756123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.756508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.756807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.757482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.761171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.837146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 881/1748] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:35 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:35.840419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.848200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d256771-2bf9-4161-bf73-0b8aac0271da None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.088215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.093155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.093552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.093872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.113465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d3f5-3569-46c7-a62d-a20eff8daa43 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.128183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.131840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.133016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.133245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.136681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.143094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.154565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.178778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.224710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 882/1749] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:35 2022] POST /identity/v3/users => generated 284 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:36.228941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.254965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 868/1750] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 853 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:36.261894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf392839-1ac8-437f-8dac-76dcec3a3537 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.284058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.286194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.286519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.286801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.287465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.289217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.370969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.385504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 883/1751] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:36.386284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea68f0-bcf3-4afb-8966-19d124d4bbc9 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.416224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56454b17-44f3-44ba-9e64-4ad048d6b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.417060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454b17-44f3-44ba-9e64-4ad048d6b81b admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.417255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454b17-44f3-44ba-9e64-4ad048d6b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.417492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454b17-44f3-44ba-9e64-4ad048d6b81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.418164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56454b17-44f3-44ba-9e64-4ad048d6b81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.438117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56454b17-44f3-44ba-9e64-4ad048d6b81b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.526052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.543221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b40362-4635-471d-a4de-2eba30d2d4cf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.543900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 869/1752] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:36.580788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24655a-ce80-4403-a8bb-cc7c1417e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.581505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24655a-ce80-4403-a8bb-cc7c1417e821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.581836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24655a-ce80-4403-a8bb-cc7c1417e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.582116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24655a-ce80-4403-a8bb-cc7c1417e821 admin admin] PATH_INFO: `/v3/projects/458d3d2f293e4f40b609f0cb2ea34c29/users/a1f9c2bc821144eea76859351eeebba4/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.682165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24655a-ce80-4403-a8bb-cc7c1417e821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=458d3d2f293e4f40b609f0cb2ea34c29, user_id=a1f9c2bc821144eea76859351eeebba4, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.702594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24655a-ce80-4403-a8bb-cc7c1417e821 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.781930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 884/1753] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:36 2022] PUT /identity/v3/projects/458d3d2f293e4f40b609f0cb2ea34c29/users/a1f9c2bc821144eea76859351eeebba4/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:36.795112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eee648-175e-48e4-a3b1-ed1d6052883e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.795526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eee648-175e-48e4-a3b1-ed1d6052883e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.795936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eee648-175e-48e4-a3b1-ed1d6052883e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.879929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.880426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.880795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.938336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-77eee648-175e-48e4-a3b1-ed1d6052883e None None] MFA Rules not processed for user `a1f9c2bc821144eea76859351eeebba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:36.939753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-77eee648-175e-48e4-a3b1-ed1d6052883e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:36.954610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:36.960025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:37.007077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.199008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 885/1754] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:30:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:37.231267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 870/1755] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:37.232091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.423348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f62478-fa2b-443b-afa4-079395afa957 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.430396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afd4b4-70f3-40a9-a093-8603372c2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.431329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afd4b4-70f3-40a9-a093-8603372c2db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.435492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afd4b4-70f3-40a9-a093-8603372c2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.435944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afd4b4-70f3-40a9-a093-8603372c2db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.436644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afd4b4-70f3-40a9-a093-8603372c2db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.438751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afd4b4-70f3-40a9-a093-8603372c2db8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.478323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39200f1-71fc-4bdd-86f8-6951a4ba3776 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.478977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39200f1-71fc-4bdd-86f8-6951a4ba3776 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.479392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39200f1-71fc-4bdd-86f8-6951a4ba3776 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.481367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 871/1756] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:37.528538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 886/1757] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:37 2022] POST /identity/v3/projects => generated 369 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:30:37.529593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39200f1-71fc-4bdd-86f8-6951a4ba3776 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.537614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.564870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.565593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.565593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.612036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:37.614692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:37.670754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39200f1-71fc-4bdd-86f8-6951a4ba3776 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.675843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.676459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.676738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.677148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.677712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.680300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.740926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 872/1758] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:37 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:37.745472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.775478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aa9ff-5b3b-49c2-98ee-41accd995827 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.798181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 887/1759] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:30:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:37.803169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.803999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.804503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.805428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.807477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.809404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.827017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 873/1760] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:30:37.831345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.880267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c67ff-0943-4d9d-9894-1d0c46947d4f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.893140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.897532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:37.899487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.900842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.901336 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.901703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] PATH_INFO: `/v3/projects/fcf6da84681341ccac8a47041f5a3404/users/db564707f75e44c8a984e4ca49dfb2ab/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.924264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e9fe3-2e59-44f7-ac0d-86bb5f7cddfb admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.928189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf6da84681341ccac8a47041f5a3404, user_id=db564707f75e44c8a984e4ca49dfb2ab, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.935071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.935524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.937946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.938403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.939480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] PATH_INFO: `/v3/projects/bfbd362ed86c407e92d4b2ecf55c45d4` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.940945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbd362ed86c407e92d4b2ecf55c45d4)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.944146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.954669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.970593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 888/1761] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:37 2022] PUT /identity/v3/projects/fcf6da84681341ccac8a47041f5a3404/users/db564707f75e44c8a984e4ca49dfb2ab/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:37.971050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.973530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.048412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbd362ed86c407e92d4b2ecf55c45d4', 'request_id': 'req-39d689da-9841-4c7b-bb44-5a52f2c08de8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.084034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbd362ed86c407e92d4b2ecf55c45d4', 'request_id': 'req-39d689da-9841-4c7b-bb44-5a52f2c08de8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.117807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbd362ed86c407e92d4b2ecf55c45d4', 'request_id': 'req-39d689da-9841-4c7b-bb44-5a52f2c08de8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.140971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] The token cache is being invalidate because project bfbd362ed86c407e92d4b2ecf55c45d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:38.142581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.144046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.146557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 874/1762] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:37 2022] DELETE /identity/v3/projects/bfbd362ed86c407e92d4b2ecf55c45d4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:38.147494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.149335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.217017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2283e-2d84-486b-a6e8-97349da40f3c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.277505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d689da-9841-4c7b-bb44-5a52f2c08de8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.307837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.308415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.308813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.309197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.309689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.315878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.322831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.330944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 875/1763] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:38.331971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.335147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.353204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4b9bd-f47a-4eab-a280-ff8851d1bba5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.355446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.356005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.356349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.356536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.357952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.358614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.358734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.359037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.359375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] PATH_INFO: `/v3/projects/24ce709f07234169802b6bcef3cde279` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.360615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=24ce709f07234169802b6bcef3cde279)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.362666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.372557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.374548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c749588-520c-4115-9da8-d2538905f182 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.397087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 889/1764] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:38.398802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.399162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.399408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.422770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.424002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.445047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24ce709f07234169802b6bcef3cde279', 'request_id': 'req-19b2362e-728b-4567-a1b6-d298fe886a2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.454625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24ce709f07234169802b6bcef3cde279', 'request_id': 'req-19b2362e-728b-4567-a1b6-d298fe886a2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.471347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24ce709f07234169802b6bcef3cde279', 'request_id': 'req-19b2362e-728b-4567-a1b6-d298fe886a2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.493004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] The token cache is being invalidate because project 24ce709f07234169802b6bcef3cde279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:38.493325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.495246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.497467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 876/1765] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:38 2022] DELETE /identity/v3/projects/24ce709f07234169802b6bcef3cde279 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:38.498277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.499875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.517399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 890/1766] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:38.518120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.519628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.536611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc0e03-8213-4237-8aff-32ae5cf22d89 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.544291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.545362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.545638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.545912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] PATH_INFO: `/v3/projects/51edea3eb09c48f2b346ae28d5b4feeb` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.547180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51edea3eb09c48f2b346ae28d5b4feeb)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.549162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.584740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.611397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2362e-728b-4567-a1b6-d298fe886a2c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.618196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51edea3eb09c48f2b346ae28d5b4feeb', 'request_id': 'req-79897df5-c70a-4bc6-a5b4-1f1ac3817849'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.629434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51edea3eb09c48f2b346ae28d5b4feeb', 'request_id': 'req-79897df5-c70a-4bc6-a5b4-1f1ac3817849'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.639523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51edea3eb09c48f2b346ae28d5b4feeb', 'request_id': 'req-79897df5-c70a-4bc6-a5b4-1f1ac3817849'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.651528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] The token cache is being invalidate because project 51edea3eb09c48f2b346ae28d5b4feeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:38.651753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.652429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:38.654084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 891/1767] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:38 2022] DELETE /identity/v3/projects/51edea3eb09c48f2b346ae28d5b4feeb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:38.654902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.656319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.701027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.701747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.702262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.702438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.704378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.719718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.721555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3e2fd-64ec-4a15-bc62-324f4462a0ff service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.732462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897df5-c70a-4bc6-a5b4-1f1ac3817849 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.732752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 877/1768] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:38.734270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5960-4e08-471c-a3d7-f088a265e175 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.734569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5960-4e08-471c-a3d7-f088a265e175 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.734895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5960-4e08-471c-a3d7-f088a265e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.747614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162928e4-105a-44fb-b8a0-decdf020750c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.748361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162928e4-105a-44fb-b8a0-decdf020750c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.749004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162928e4-105a-44fb-b8a0-decdf020750c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.749329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162928e4-105a-44fb-b8a0-decdf020750c admin admin] PATH_INFO: `/v3/projects/fcf6da84681341ccac8a47041f5a3404/users/db564707f75e44c8a984e4ca49dfb2ab/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.753052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.753557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162928e4-105a-44fb-b8a0-decdf020750c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf6da84681341ccac8a47041f5a3404, user_id=db564707f75e44c8a984e4ca49dfb2ab, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.757401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162928e4-105a-44fb-b8a0-decdf020750c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.761603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6dea5960-4e08-471c-a3d7-f088a265e175 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.763461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6dea5960-4e08-471c-a3d7-f088a265e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.773189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 892/1769] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:38 2022] PUT /identity/v3/projects/fcf6da84681341ccac8a47041f5a3404/users/db564707f75e44c8a984e4ca49dfb2ab/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:38.774349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35fc85-51fa-4197-886c-92abfcd6a3f4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.774734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35fc85-51fa-4197-886c-92abfcd6a3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.775487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35fc85-51fa-4197-886c-92abfcd6a3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.814762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8e35fc85-51fa-4197-886c-92abfcd6a3f4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.816269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8e35fc85-51fa-4197-886c-92abfcd6a3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.856188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 878/1770] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:38.862192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c336-eed0-4d19-8d9c-7c79c6554255 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.862502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c336-eed0-4d19-8d9c-7c79c6554255 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.862784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c336-eed0-4d19-8d9c-7c79c6554255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.886598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ca68c336-eed0-4d19-8d9c-7c79c6554255 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.888856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ca68c336-eed0-4d19-8d9c-7c79c6554255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.903557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 879/1771] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:38.904905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.905134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.905409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.912117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 893/1772] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:38.912117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184b067-3b9e-4bab-8584-819f34841e8a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.912117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184b067-3b9e-4bab-8584-819f34841e8a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.912117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184b067-3b9e-4bab-8584-819f34841e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.943696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9184b067-3b9e-4bab-8584-819f34841e8a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.946322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] MFA Rules not processed for user `db564707f75e44c8a984e4ca49dfb2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.946800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9184b067-3b9e-4bab-8584-819f34841e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.948174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:38.960552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 894/1773] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:38.963192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.963756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.964460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.996072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:38.997799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:39.012573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 895/1774] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:39.014001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.028108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768341b3-9476-42f6-bddb-448a466e17cc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.033187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.034221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.034808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.035233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.035913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:39.036214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.038134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.053903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 896/1775] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:39.054865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.062231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 880/1776] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:39.062993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.078207 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7547eb-86c9-45c0-ad90-ae67167a217f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.083816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e468dc2-3b6b-4ac3-bfef-c425c1c9f703 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.089424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.090455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.090736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.091009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.091663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:39.091963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.095154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.095305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.096255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.096703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.097174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.097961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:39.098396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.103761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.110434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 881/1777] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:39.111283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.121973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 897/1778] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:39.123009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.128610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:39.129047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633283f-add2-4b56-91f5-6fe0e9bbc81f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.135883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.136649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.137054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.137403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.138113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.139592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.140395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.188480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 882/1779] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:39.189728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.219374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a044c-74d1-454b-bbe8-aaa596898416 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.265511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.295068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f69238-4388-4a5a-8706-d59da9bc4370 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.306341 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.308088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.308522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.309004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.309653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.312163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.327435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.328067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.328320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.328595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.351500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 883/1780] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/projects => generated 403 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:39.356442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.421484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.423081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.435207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d34c6-8d64-4b89-9686-a461ca785a72 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.437087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 898/1781] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:39.438440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.450855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d2946-de4f-427d-9019-eb268d071e4f service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.451813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.453508 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.453801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.454173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.454897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.455978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.456884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.457009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.457273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.458272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.459162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.461587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.492737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 899/1782] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/users => generated 294 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:39.492737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.501661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 884/1783] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:39.505992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.508311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e4a7-350b-49e9-ad59-9c1d6295586b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.624074 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.644639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.646438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.646924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.647956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.648989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.651419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c213-f58e-4814-9857-f742433766eb admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.653235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.673188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.674338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.674619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.674944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.677657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.691169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.708681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 885/1784] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:39.708681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.717469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 900/1785] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/users => generated 302 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:39.723248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.723479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ad4d9-e6de-4b0f-8947-edc8e0c9055e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.728800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.729497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.729785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.730061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.730704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.733318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.742024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45be676-9992-4127-a052-5e3b4291ba15 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.762884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.763621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.763845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.764100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.764677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.772388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.798142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 886/1786] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:39.798820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.815285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 901/1787] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:39.816495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.818202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a1861-b226-4282-bc53-147c4fe6deb8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.826095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.826908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.827207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.827464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.829264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.832880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.854080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7603a1-885a-4287-8765-e7a59a66966c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.858380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 887/1788] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:39.862863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.871668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.872451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.872729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.873032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.873934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.875567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.888242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc11f6f-583e-44e3-a73a-b10aa81f83a2 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.900154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.902438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.902438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.902438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] PATH_INFO: `/v3/projects/f0250cd932174cd89af7783e796ae519/users/8a982c729a82443e9636c5a8f96b657d/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.912965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 902/1789] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:39.913878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.927389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0250cd932174cd89af7783e796ae519, user_id=8a982c729a82443e9636c5a8f96b657d, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.930860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.939482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a46a8-2090-499e-89e3-f63a0cd22501 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.946186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd5b21-858a-4aaf-9af9-e9b4fdee9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.947039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5b21-858a-4aaf-9af9-e9b4fdee9877 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.948333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5b21-858a-4aaf-9af9-e9b4fdee9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.949871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5b21-858a-4aaf-9af9-e9b4fdee9877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.950861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd5b21-858a-4aaf-9af9-e9b4fdee9877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.953647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd5b21-858a-4aaf-9af9-e9b4fdee9877 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.957479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 888/1790] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:39 2022] PUT /identity/v3/projects/f0250cd932174cd89af7783e796ae519/users/8a982c729a82443e9636c5a8f96b657d/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:39.958927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.988335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 903/1791] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:39.988575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.993412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.995167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.995648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.040476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] MFA Rules not processed for user `8a982c729a82443e9636c5a8f96b657d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:40.043567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:40.078395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841c41-aa4c-46d4-ab94-3059c32339fa admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.085127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92779fec-db91-426f-a617-a339e173414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.086398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92779fec-db91-426f-a617-a339e173414b admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.086912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92779fec-db91-426f-a617-a339e173414b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.088244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92779fec-db91-426f-a617-a339e173414b admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.089705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92779fec-db91-426f-a617-a339e173414b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:40.092271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92779fec-db91-426f-a617-a339e173414b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:40.115452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 889/1792] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:40.116678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92779fec-db91-426f-a617-a339e173414b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:40.142449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92779fec-db91-426f-a617-a339e173414b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.162024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.162897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.163184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.163463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] PATH_INFO: `/v3/projects/1c43f2faabb842d4b37c8beb665c136e/users/b101aee0dfc54afbb9814914061a0e2c/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.188666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 904/1793] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:40.189710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:40.206057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55285a-d61a-4fe6-a946-c68116168b60 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.217287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bbf28-40ea-408d-974f-a30de2218b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.219670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bbf28-40ea-408d-974f-a30de2218b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.221096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bbf28-40ea-408d-974f-a30de2218b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.221502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bbf28-40ea-408d-974f-a30de2218b38 admin admin] PATH_INFO: `/v3/projects/63ab476760f642c7a4d177c5e1ff0629/users/435906d7a5d847749aa5f3069c1def41/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.228248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c43f2faabb842d4b37c8beb665c136e, user_id=b101aee0dfc54afbb9814914061a0e2c, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:40.233178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:40.268694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 890/1794] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:40 2022] PUT /identity/v3/projects/1c43f2faabb842d4b37c8beb665c136e/users/b101aee0dfc54afbb9814914061a0e2c/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:40.273158 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:40.273875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bbf28-40ea-408d-974f-a30de2218b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ab476760f642c7a4d177c5e1ff0629, user_id=435906d7a5d847749aa5f3069c1def41, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:40.286148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bbf28-40ea-408d-974f-a30de2218b38 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:40.360815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 905/1795] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:40 2022] PUT /identity/v3/projects/63ab476760f642c7a4d177c5e1ff0629/users/435906d7a5d847749aa5f3069c1def41/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:40.365592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a514130-3049-4755-a27f-3f4e86837eb2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.366473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a514130-3049-4755-a27f-3f4e86837eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.367115 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a514130-3049-4755-a27f-3f4e86837eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.501849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0a514130-3049-4755-a27f-3f4e86837eb2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:40.512864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470536e6-39b2-444e-a45e-0ab9af053d3e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.520734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0a514130-3049-4755-a27f-3f4e86837eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:40.863815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 906/1796] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:40.864257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.864963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.865332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.865610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.866914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.870019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:40.870316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.870805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.872133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:40.976976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:40.982687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] MFA Rules not processed for user `435906d7a5d847749aa5f3069c1def41`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:40.984814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:40.986466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 891/1797] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:40 2022] POST /identity/v3/projects => generated 387 bytes in 717 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:40.991170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.026132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:41.215360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1f5a2-4a8a-4582-b7d1-1c495b334aa0 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.227687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 907/1798] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:41.227687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.233891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.234476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.234764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.240041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:41.245153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.245911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.275965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.374288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 892/1799] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:40 2022] POST /identity/v3/users => generated 300 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:41.375612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.414417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44ebfc-724d-4523-8be9-5124bb524e32 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.434696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.435659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.436356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.436862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.444865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.447700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.534138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02148f-080f-4bfb-b79e-719af56c03f2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.550530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 893/1800] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:41 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:41.551505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.553349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.553980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.554309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.554520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.587628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.591033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.624634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 908/1801] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:41.625592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.647459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912c4db-d2c4-402e-b9c7-72287e64a2d4 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.777874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a982-1d6c-4d5e-96eb-80dc7248c8c9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.813618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.814753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.815350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.817912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.819520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.826243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.858033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 894/1802] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:41.868078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.902189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.902189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.902189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.902189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.902856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.908056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa51597-955d-47ce-8fc9-aed51e9cc2fa admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.908237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.946705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.948673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.951451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.951897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] PATH_INFO: `/v3/projects/693f3095819049fe89df6222c09deec2/users/9a21403b6b7f4ad1b51eb81b6d4d456a/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.995765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 909/1803] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:41 2022] POST /identity/v3/users => generated 299 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:41.999939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.031395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e54dc0-b7a6-42e4-ace5-d6c0b9e8eafc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.039249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693f3095819049fe89df6222c09deec2, user_id=9a21403b6b7f4ad1b51eb81b6d4d456a, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.049159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.049596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.049724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.050024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.050266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.052182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.056359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.091916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 895/1804] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:41 2022] PUT /identity/v3/projects/693f3095819049fe89df6222c09deec2/users/9a21403b6b7f4ad1b51eb81b6d4d456a/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:42.098246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.105839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 910/1805] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:42.106780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.131081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e4078-54f1-4165-a77d-460229a2f435 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.220046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.354895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.355629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.355908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.356178 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] PATH_INFO: `/v3/projects/ff4751e036184a4daef7bc1c624616b7/users/aad7952d4fad419bb12971e2856fdde5/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.377641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.426647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4751e036184a4daef7bc1c624616b7, user_id=aad7952d4fad419bb12971e2856fdde5, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.438750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.482638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc9fc3-f227-4686-a42c-ef4274c102d0 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.499382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 911/1806] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:42 2022] PUT /identity/v3/projects/ff4751e036184a4daef7bc1c624616b7/users/aad7952d4fad419bb12971e2856fdde5/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:42.506248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.636353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.638059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.638059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.638059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.639293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.641163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.672738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 896/1807] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 575 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:42.674498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.701900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c9293-5655-42de-8afd-067395e8da5e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.721059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ac907-ac64-45ba-a5c6-e9f839d1631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.723558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ac907-ac64-45ba-a5c6-e9f839d1631e admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.723894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ac907-ac64-45ba-a5c6-e9f839d1631e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.725044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ac907-ac64-45ba-a5c6-e9f839d1631e admin admin] PATH_INFO: `/v3/projects/693f3095819049fe89df6222c09deec2/users/9a21403b6b7f4ad1b51eb81b6d4d456a/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.749378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ac907-ac64-45ba-a5c6-e9f839d1631e admin admin] RBAC: Authorizing `identity:create_grant(project_id=693f3095819049fe89df6222c09deec2, user_id=9a21403b6b7f4ad1b51eb81b6d4d456a, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.760239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ac907-ac64-45ba-a5c6-e9f839d1631e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.787567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d9085-f289-4f64-a6ca-6d91daf97614 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.821994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 897/1808] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:42 2022] PUT /identity/v3/projects/693f3095819049fe89df6222c09deec2/users/9a21403b6b7f4ad1b51eb81b6d4d456a/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:42.829475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.830228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.830228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.965654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.966612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.967268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.967733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.968578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.970424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.973565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] MFA Rules not processed for user `9a21403b6b7f4ad1b51eb81b6d4d456a`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:42.980583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:43.005108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 912/1809] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:43.006047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.045924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e9cff-5d15-4af5-86d9-4a5c5c526d39 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.058141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b4132-5f32-4a1b-bf4c-ebc803a0d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.058857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4132-5f32-4a1b-bf4c-ebc803a0d3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.059140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4132-5f32-4a1b-bf4c-ebc803a0d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.059420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4132-5f32-4a1b-bf4c-ebc803a0d3c6 admin admin] PATH_INFO: `/v3/projects/ff4751e036184a4daef7bc1c624616b7/users/aad7952d4fad419bb12971e2856fdde5/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.063654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b4132-5f32-4a1b-bf4c-ebc803a0d3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4751e036184a4daef7bc1c624616b7, user_id=aad7952d4fad419bb12971e2856fdde5, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.070143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b4132-5f32-4a1b-bf4c-ebc803a0d3c6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.117244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 913/1810] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:43 2022] PUT /identity/v3/projects/ff4751e036184a4daef7bc1c624616b7/users/aad7952d4fad419bb12971e2856fdde5/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:43.117471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.118172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.118172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.162202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] MFA Rules not processed for user `aad7952d4fad419bb12971e2856fdde5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:43.164367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:43.266515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.376405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 914/1811] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:43.383921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.406552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 898/1812] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 584 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:43.407836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.424135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de70e6-1dfa-4117-9459-e477773792b7 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.578589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97ba1-e527-4932-857c-22e7b36dfe74 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.591575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.591575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.591879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.592054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.598675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.599415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.599769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.600110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] PATH_INFO: `/v3/users/a1f9c2bc821144eea76859351eeebba4` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.602179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f9c2bc821144eea76859351eeebba4)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.604388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.610397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.645204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.647606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.665263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 915/1813] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:43.667477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.669195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:43.686012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352c28c-83b6-4919-aac2-adcc3eaddfb2 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.687906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f9c2bc821144eea76859351eeebba4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:43.691840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31ce00-e900-409c-b31b-c782f4b7a7af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.692675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31ce00-e900-409c-b31b-c782f4b7a7af service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.693227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31ce00-e900-409c-b31b-c782f4b7a7af service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.693777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31ce00-e900-409c-b31b-c782f4b7a7af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.697308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f9c2bc821144eea76859351eeebba4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:43.698575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f9c2bc821144eea76859351eeebba4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:43.716872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f9c2bc821144eea76859351eeebba4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:43.752082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f9c2bc821144eea76859351eeebba4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:43.776219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 899/1814] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:43 2022] DELETE /identity/v3/users/a1f9c2bc821144eea76859351eeebba4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:43.777673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.779316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:43.838194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31ce00-e900-409c-b31b-c782f4b7a7af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.839421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31ce00-e900-409c-b31b-c782f4b7a7af service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.856323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 916/1815] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:43.858162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.859684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.860075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.882333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:43.884325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:43.950167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8120da1-be17-42d2-bad0-0a3f73242844 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.965959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.984808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.986721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.987822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] PATH_INFO: `/v3/users/435906d7a5d847749aa5f3069c1def41` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.999735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=435906d7a5d847749aa5f3069c1def41)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.014959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.016980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.036624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 917/1816] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:44.039904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.039904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.059374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0f289-a827-43f4-bed1-015e7053d074 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.071052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.071752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.072032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.072334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] PATH_INFO: `/v3/users/8a982c729a82443e9636c5a8f96b657d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.073523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a982c729a82443e9636c5a8f96b657d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.075464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.165501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a982c729a82443e9636c5a8f96b657d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.184678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a982c729a82443e9636c5a8f96b657d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.199901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435906d7a5d847749aa5f3069c1def41'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.221863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a982c729a82443e9636c5a8f96b657d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.223472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a982c729a82443e9636c5a8f96b657d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.227667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435906d7a5d847749aa5f3069c1def41'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.229175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435906d7a5d847749aa5f3069c1def41'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.238702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a982c729a82443e9636c5a8f96b657d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.240389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435906d7a5d847749aa5f3069c1def41'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.247245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 918/1817] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:44 2022] DELETE /identity/v3/users/8a982c729a82443e9636c5a8f96b657d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:44.255733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.263903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.267635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b0a1da60-f3cf-4a52-b5a2-43923f214991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435906d7a5d847749aa5f3069c1def41'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:44.295401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 900/1818] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:43 2022] DELETE /identity/v3/users/435906d7a5d847749aa5f3069c1def41 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:44.297781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0953-187e-4f86-b45d-3b1a623f3a69 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.298175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0953-187e-4f86-b45d-3b1a623f3a69 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.298520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0953-187e-4f86-b45d-3b1a623f3a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.316965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.346667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-152a0953-187e-4f86-b45d-3b1a623f3a69 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:44.347915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-152a0953-187e-4f86-b45d-3b1a623f3a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.383257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2d49c-4744-421e-9385-4fd96c3bb2bc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.442700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 901/1819] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:44.444788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f9b0f-9c9d-428f-a77d-26b398dfcd83 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.445178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f9b0f-9c9d-428f-a77d-26b398dfcd83 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.445553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f9b0f-9c9d-428f-a77d-26b398dfcd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.471980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-635f9b0f-9c9d-428f-a77d-26b398dfcd83 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:44.473619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-635f9b0f-9c9d-428f-a77d-26b398dfcd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.482688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 902/1820] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:44.492808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.493451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.493731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.494004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.496058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.496544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.497265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.500312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.522939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.524392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.530825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] MFA Rules not processed for user `b101aee0dfc54afbb9814914061a0e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:44.532109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.534688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 919/1821] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:44.535507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.536750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.546987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ffdcb-d52e-460a-a789-abfba0304ea1 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.551214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.551814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.552149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.552986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.567703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.568980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.589051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 920/1822] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:44.590269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.601276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18169e9c-ecca-440e-a0af-25854daf4fc4 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.605946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.607662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.607662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.607845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.624636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.626119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.642465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 903/1823] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:44.644670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.646220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:44.648642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 921/1824] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:44.669825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106c1a1-fb6e-41d5-ae44-4fd5a4c6ebd8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.683392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.684074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.684684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.685153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.685852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.689250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.737175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 904/1825] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:44 2022] POST /identity/v3/projects => generated 403 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:44.741385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.749399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.754682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a76e-9799-440e-b10f-55ebb57b7edc service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.898801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.899871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.901759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.902119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.902778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.904827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.950880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 922/1826] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:44 2022] POST /identity/v3/users => generated 308 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:30:44.953077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.973779 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7947597-5574-412c-a40c-df55b096de5a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.993464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.994449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.996347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.999931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.001516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.006865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.047647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 905/1827] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:45.051605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.065125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8d409-40da-4b99-b53b-c46d0480211f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.071263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.072126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.075065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.075544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] PATH_INFO: `/v3/projects/90b16c7c657744739fceeb943704aadb/users/0e97d47baa8c4ee28cf48098885ada2d/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.140130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b16c7c657744739fceeb943704aadb, user_id=0e97d47baa8c4ee28cf48098885ada2d, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.143581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.169373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 923/1828] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:45 2022] PUT /identity/v3/projects/90b16c7c657744739fceeb943704aadb/users/0e97d47baa8c4ee28cf48098885ada2d/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:45.174019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.187470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1448ed-1f49-45f6-a2bd-b86b4cb03e4f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.244862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.247029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:45.306690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.320887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.321733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.322083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.323147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.324580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.326384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.344557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 906/1829] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:45.348817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.367954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92e2bf-a77a-4165-87e9-53fb3663d69c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.378475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc1cda-87ae-4862-b06a-cfb0c47c1594 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.378663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.379690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.380078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.380547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] PATH_INFO: `/v3/projects/90b16c7c657744739fceeb943704aadb/users/0e97d47baa8c4ee28cf48098885ada2d/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.383706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.384468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.384746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.386043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] PATH_INFO: `/v3/projects/63ab476760f642c7a4d177c5e1ff0629` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.388819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ab476760f642c7a4d177c5e1ff0629)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.389309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b16c7c657744739fceeb943704aadb, user_id=0e97d47baa8c4ee28cf48098885ada2d, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.391167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.394173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.412276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 907/1830] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:45 2022] PUT /identity/v3/projects/90b16c7c657744739fceeb943704aadb/users/0e97d47baa8c4ee28cf48098885ada2d/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:45.559086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ab476760f642c7a4d177c5e1ff0629', 'request_id': 'req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:45.567865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.578133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:45.601444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ab476760f642c7a4d177c5e1ff0629', 'request_id': 'req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:45.628915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ab476760f642c7a4d177c5e1ff0629', 'request_id': 'req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:45.642592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.652645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] The token cache is being invalidate because project 63ab476760f642c7a4d177c5e1ff0629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:45.653437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:45.655726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:45.664871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 924/1831] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:45 2022] DELETE /identity/v3/projects/63ab476760f642c7a4d177c5e1ff0629 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:45.677620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.686908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:45.871769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd4402-7a03-4da3-8028-91a2ece4074f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.901738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaf8cd-67b2-4acb-8136-06f0e28073fd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.933599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.935532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.935986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.937631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] PATH_INFO: `/v3/projects/458d3d2f293e4f40b609f0cb2ea34c29` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.940641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=458d3d2f293e4f40b609f0cb2ea34c29)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.945272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.967588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.970035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.971100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.972381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] PATH_INFO: `/v3/projects/f0250cd932174cd89af7783e796ae519` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.976661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0250cd932174cd89af7783e796ae519)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.978963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.012152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458d3d2f293e4f40b609f0cb2ea34c29', 'request_id': 'req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.019010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458d3d2f293e4f40b609f0cb2ea34c29', 'request_id': 'req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.034219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458d3d2f293e4f40b609f0cb2ea34c29', 'request_id': 'req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.065976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] The token cache is being invalidate because project 458d3d2f293e4f40b609f0cb2ea34c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:46.066541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.069953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.071524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 925/1832] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:45 2022] DELETE /identity/v3/projects/458d3d2f293e4f40b609f0cb2ea34c29 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:46.075172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.075172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:46.119157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0250cd932174cd89af7783e796ae519', 'request_id': 'req-bcfb700c-0dbc-4085-86a7-4098507a8d23'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.146074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0250cd932174cd89af7783e796ae519', 'request_id': 'req-bcfb700c-0dbc-4085-86a7-4098507a8d23'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.166423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0250cd932174cd89af7783e796ae519', 'request_id': 'req-bcfb700c-0dbc-4085-86a7-4098507a8d23'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.190797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] The token cache is being invalidate because project f0250cd932174cd89af7783e796ae519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:46.191082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.192150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.193626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 908/1833] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:45 2022] DELETE /identity/v3/projects/f0250cd932174cd89af7783e796ae519 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:46.194414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.195722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:46.202422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6a4c-1e57-4aa9-ba0e-5fc5d76354fc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.210953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.256851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.288437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.289836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.290121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.290399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] PATH_INFO: `/v3/users/aad7952d4fad419bb12971e2856fdde5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.292908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad7952d4fad419bb12971e2856fdde5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.295212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.313095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb700c-0dbc-4085-86a7-4098507a8d23 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.336231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.336967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.338614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.338614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] PATH_INFO: `/v3/users/db564707f75e44c8a984e4ca49dfb2ab` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.340647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db564707f75e44c8a984e4ca49dfb2ab)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.343407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.379505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad7952d4fad419bb12971e2856fdde5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.391937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad7952d4fad419bb12971e2856fdde5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.410592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad7952d4fad419bb12971e2856fdde5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.411504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad7952d4fad419bb12971e2856fdde5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.421132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db564707f75e44c8a984e4ca49dfb2ab'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.424446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad7952d4fad419bb12971e2856fdde5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.430116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db564707f75e44c8a984e4ca49dfb2ab'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.431429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db564707f75e44c8a984e4ca49dfb2ab'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.434851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 926/1834] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:46 2022] DELETE /identity/v3/users/aad7952d4fad419bb12971e2856fdde5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:46.435696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.437092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:46.459164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db564707f75e44c8a984e4ca49dfb2ab'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.488533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-00928fde-6975-4842-9c61-738d7a269aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db564707f75e44c8a984e4ca49dfb2ab'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:46.508007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 909/1835] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:46 2022] DELETE /identity/v3/users/db564707f75e44c8a984e4ca49dfb2ab => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:46.509486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.509770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.518449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.601038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] MFA Rules not processed for user `0e97d47baa8c4ee28cf48098885ada2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:46.603187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:46.657814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ae624-bfe5-4533-84a7-327c4b4ea87e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.797096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 910/1836] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:46 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:46.812284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.813895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.814325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.814750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] PATH_INFO: `/v3/users/9a21403b6b7f4ad1b51eb81b6d4d456a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.816917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a21403b6b7f4ad1b51eb81b6d4d456a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.819642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.828279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.986366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a21403b6b7f4ad1b51eb81b6d4d456a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:47.007102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a21403b6b7f4ad1b51eb81b6d4d456a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:47.063364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a21403b6b7f4ad1b51eb81b6d4d456a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:47.065127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a21403b6b7f4ad1b51eb81b6d4d456a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:47.078092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d5117417-a694-422c-b076-6065613ee2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a21403b6b7f4ad1b51eb81b6d4d456a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:47.091243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 927/1837] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:46 2022] DELETE /identity/v3/users/9a21403b6b7f4ad1b51eb81b6d4d456a => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:47.799188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:47.801061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:47.836876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90707-d633-46ad-aa2b-e5f08085650a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.861311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.942853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.943618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.944402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.945096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] PATH_INFO: `/v3/projects/ff4751e036184a4daef7bc1c624616b7` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.946419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4751e036184a4daef7bc1c624616b7)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:47.949340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.012075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4751e036184a4daef7bc1c624616b7', 'request_id': 'req-5e843739-ddd2-4c39-aae2-aa84a5c4756a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.020969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4751e036184a4daef7bc1c624616b7', 'request_id': 'req-5e843739-ddd2-4c39-aae2-aa84a5c4756a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.042905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4751e036184a4daef7bc1c624616b7', 'request_id': 'req-5e843739-ddd2-4c39-aae2-aa84a5c4756a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.059042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] The token cache is being invalidate because project ff4751e036184a4daef7bc1c624616b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:48.059615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.060888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5e843739-ddd2-4c39-aae2-aa84a5c4756a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.062746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 911/1838] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:47 2022] DELETE /identity/v3/projects/ff4751e036184a4daef7bc1c624616b7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:48.068292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.069244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.069244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.099137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:48.100588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.168357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 928/1839] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:48.177322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.177693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.178034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.199531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:48.201506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.208901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 912/1840] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:48.213733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.224222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c97e4-fa60-4cd8-9a9e-f95f1868626f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.228739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.230284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.230605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.230877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.231558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:48.231855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.233503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.246436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 929/1841] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:48.250968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.261153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c8d91-9dff-4aee-a2e5-b31aa89fe120 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.265896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.266799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.266799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.267021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.267745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.270363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.305504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.307825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.319787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 913/1842] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:48.320711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.361565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76e82e-c53b-48d4-a470-e3bade4cf74c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.416010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2ce41-2f0b-4c64-bd43-49a41650dac5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.424784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb594db0-7535-4e04-95c3-45caa302030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.426022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb594db0-7535-4e04-95c3-45caa302030a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.426384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb594db0-7535-4e04-95c3-45caa302030a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.426974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb594db0-7535-4e04-95c3-45caa302030a admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.427837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb594db0-7535-4e04-95c3-45caa302030a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.433903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb594db0-7535-4e04-95c3-45caa302030a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.447288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.448198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.448651 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.449025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] PATH_INFO: `/v3/projects/fcf6da84681341ccac8a47041f5a3404` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.450394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf6da84681341ccac8a47041f5a3404)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.453524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.474385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 914/1843] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:48.474385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd427de-b04c-4b30-b35c-888614efd5c2 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.474385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd427de-b04c-4b30-b35c-888614efd5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.474385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd427de-b04c-4b30-b35c-888614efd5c2 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.475086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 915/1844] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:48.475690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd427de-b04c-4b30-b35c-888614efd5c2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.476250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:48.497798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd427de-b04c-4b30-b35c-888614efd5c2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.504035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f3d0-8349-4eb7-a486-5032958700de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.504652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f3d0-8349-4eb7-a486-5032958700de admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.504964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f3d0-8349-4eb7-a486-5032958700de admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.505862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f3d0-8349-4eb7-a486-5032958700de admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.506525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f3d0-8349-4eb7-a486-5032958700de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.508143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f3d0-8349-4eb7-a486-5032958700de admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.512299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:48.531747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 916/1845] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:48.531747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8302-dd63-48d1-a929-39ee70c108bb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.531747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8302-dd63-48d1-a929-39ee70c108bb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.531747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8302-dd63-48d1-a929-39ee70c108bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.558542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-020f8302-dd63-48d1-a929-39ee70c108bb None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:48.560512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-020f8302-dd63-48d1-a929-39ee70c108bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.564379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf6da84681341ccac8a47041f5a3404', 'request_id': 'req-60084c59-f842-4dfa-b16e-87bc1f9c4910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.578453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf6da84681341ccac8a47041f5a3404', 'request_id': 'req-60084c59-f842-4dfa-b16e-87bc1f9c4910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.587334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf6da84681341ccac8a47041f5a3404', 'request_id': 'req-60084c59-f842-4dfa-b16e-87bc1f9c4910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.617247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] The token cache is being invalidate because project fcf6da84681341ccac8a47041f5a3404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:48.618089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.619380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:48.626051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 930/1846] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:48 2022] DELETE /identity/v3/projects/fcf6da84681341ccac8a47041f5a3404 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:48.626971 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.630480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.688383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 917/1847] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:48.691409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.691855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.693031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.740702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:48.742587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.766778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 918/1848] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:48.767981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.770666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60084c59-f842-4dfa-b16e-87bc1f9c4910 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.772214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.801637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.802290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.802600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.802831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.803490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.807935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.815032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0f6cd-1166-434c-8bca-13648598620b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.828819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 931/1849] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:48.828819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.833579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bb9ac-da59-4a72-801c-e14949626b2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.834212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bb9ac-da59-4a72-801c-e14949626b2f service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.834961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bb9ac-da59-4a72-801c-e14949626b2f service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.835260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bb9ac-da59-4a72-801c-e14949626b2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.837068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.859783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb60265-1d92-431f-b930-7ec739112613 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.867260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.868132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.868363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bb9ac-da59-4a72-801c-e14949626b2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.870319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.870709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] PATH_INFO: `/v3/projects/693f3095819049fe89df6222c09deec2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.872728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=693f3095819049fe89df6222c09deec2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.874218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bb9ac-da59-4a72-801c-e14949626b2f service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.875645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.915120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 919/1850] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:48.921958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.922393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.922719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.966211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:48.967444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.997398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693f3095819049fe89df6222c09deec2', 'request_id': 'req-9a0fe36a-9896-43df-93e5-94b97910ac59'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:49.008914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693f3095819049fe89df6222c09deec2', 'request_id': 'req-9a0fe36a-9896-43df-93e5-94b97910ac59'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:49.021999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693f3095819049fe89df6222c09deec2', 'request_id': 'req-9a0fe36a-9896-43df-93e5-94b97910ac59'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:49.040983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 920/1851] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:49.041953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.052392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0df2f7-a670-4a34-b2f4-dbd70a4e6d7b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.058053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.058811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.059057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.059332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] PATH_INFO: `/v3/projects/d2c1caad11964d5d9ac9222707190c78/users/b94b9f8a83374c108bc80083213aea2f/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.060877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] The token cache is being invalidate because project 693f3095819049fe89df6222c09deec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:49.061181 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:49.061899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:49.063515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 932/1852] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:48 2022] DELETE /identity/v3/projects/693f3095819049fe89df6222c09deec2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:49.064433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.066072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:49.082364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fe36a-9896-43df-93e5-94b97910ac59 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.089527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.090951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.091449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.093552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.095459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:49.101925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.105230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ea423-9151-410b-9f32-07c3cfd1238d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.105629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c1caad11964d5d9ac9222707190c78, user_id=b94b9f8a83374c108bc80083213aea2f, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.108655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.134688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 933/1853] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:49.134688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47b4b-612d-461b-9d6d-7811e1a38b1c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.134688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47b4b-612d-461b-9d6d-7811e1a38b1c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.134688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47b4b-612d-461b-9d6d-7811e1a38b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.145048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 921/1854] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:49 2022] PUT /identity/v3/projects/d2c1caad11964d5d9ac9222707190c78/users/b94b9f8a83374c108bc80083213aea2f/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:49.146002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.157556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.170667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a5f47b4b-612d-461b-9d6d-7811e1a38b1c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:49.172184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a5f47b4b-612d-461b-9d6d-7811e1a38b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:49.216463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.235932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e262a-f171-45b1-b156-f393cac848f1 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.244030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1575080-61e6-4894-be84-7a98dba607a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.246199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1575080-61e6-4894-be84-7a98dba607a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.246744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1575080-61e6-4894-be84-7a98dba607a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.247120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1575080-61e6-4894-be84-7a98dba607a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.247879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1575080-61e6-4894-be84-7a98dba607a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.251962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1575080-61e6-4894-be84-7a98dba607a5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.287623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 934/1855] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:49.289462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.289748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.290017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.315665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 922/1856] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:49.318622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.318939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.319185 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.371758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:49.373015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:49.389515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] MFA Rules not processed for user `b94b9f8a83374c108bc80083213aea2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:49.391272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:49.452828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 923/1857] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:49.452828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.461912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48068a-8170-4ab4-989d-39a647d464e4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.467334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.468073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.468373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.469060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.469694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.471653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.485514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 935/1858] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:49.486519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.498160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aa898-9f40-45d1-93ec-a87531c6794c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.502795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 924/1859] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:49.502795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.504820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:49.506105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.506815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.507206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.507575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.508329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:49.508723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.511328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.518829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e814ec-ff75-4e53-ad98-af019af12834 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.529040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 936/1860] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:49.529253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.529376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.529483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.529483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.529483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.529483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.530665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.537528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7e9bd-a80f-4494-b577-c5bfe9490570 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.543518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.544235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.544635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.545048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.545824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.547563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.562479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 937/1861] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:49.563653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.567363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 925/1862] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:49.568220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.576537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02563bd-f134-40d1-ba93-923b4366b8ba admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.642337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7d633-a56c-4058-af50-5ba8855cd780 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.646568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.647403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.647534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.647713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.647991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.648150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.648540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.648658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.649297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.649474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.650722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.651701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.665931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 926/1863] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:49.665931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.673816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705788d2-7d3c-4add-9f80-259e5b86c31e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.677984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.678891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.679588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.679900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.680201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.680871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.684033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.693184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 938/1864] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:49.693184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.715741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 927/1865] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:49.715741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.726717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381edd48-85cc-4a97-9d8d-fe6a9486d239 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.740449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.770967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a026b-b2d7-41ed-8137-a98038b8f664 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.776740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.777444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.777721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.777993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] PATH_INFO: `/v3/projects/a1557b629ebc4626a836ba83ff86c453/users/d833e588bc28485db7e1af8c96753cb6/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.794755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc8af3-6ac0-4725-a5e8-8417bb2c4824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.795060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1557b629ebc4626a836ba83ff86c453, user_id=d833e588bc28485db7e1af8c96753cb6, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.795514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8af3-6ac0-4725-a5e8-8417bb2c4824 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.795888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8af3-6ac0-4725-a5e8-8417bb2c4824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.796255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8af3-6ac0-4725-a5e8-8417bb2c4824 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.797013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc8af3-6ac0-4725-a5e8-8417bb2c4824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.797804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.799312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc8af3-6ac0-4725-a5e8-8417bb2c4824 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.814630 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 939/1866] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:49 2022] PUT /identity/v3/projects/a1557b629ebc4626a836ba83ff86c453/users/d833e588bc28485db7e1af8c96753cb6/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:49.815575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.833954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 928/1867] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:49.850033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.850033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.850318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.900028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] MFA Rules not processed for user `d833e588bc28485db7e1af8c96753cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:49.901860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:49.931254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574a2e1-50a7-438f-962e-58d3e2a0dbf8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.938108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.938781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.939113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.939397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.940106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.950452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.962333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 940/1868] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:49.962998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.976215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c574df0-1a27-407b-bd3a-11aadca49a26 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.982719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6724a-e3f7-4cfa-9652-00da369f6d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.983466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6724a-e3f7-4cfa-9652-00da369f6d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.983741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6724a-e3f7-4cfa-9652-00da369f6d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.984134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6724a-e3f7-4cfa-9652-00da369f6d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.985033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6724a-e3f7-4cfa-9652-00da369f6d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.987137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6724a-e3f7-4cfa-9652-00da369f6d49 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.998045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 941/1869] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:50.000092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 929/1870] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:50.005276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.005681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99b15-69b2-432e-80be-4e4bbeb6ec13 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.005979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99b15-69b2-432e-80be-4e4bbeb6ec13 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.006255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f99b15-69b2-432e-80be-4e4bbeb6ec13 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.007237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.007845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 942/1871] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:50.008560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f99b15-69b2-432e-80be-4e4bbeb6ec13 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.022364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb45c5-6f5f-450b-b8f3-defc358f4f90 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.023489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f99b15-69b2-432e-80be-4e4bbeb6ec13 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.028007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.028870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.029152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.029433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] PATH_INFO: `/v3/users/b101aee0dfc54afbb9814914061a0e2c` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.030239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.030557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b101aee0dfc54afbb9814914061a0e2c)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.031013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.031365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.031713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.032482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.032590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.034487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.046487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 943/1872] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:50.047326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.057310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629040e-11d9-43c9-966c-c246894bce37 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.062292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.062873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.063184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.063426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.064155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.065876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.080433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 944/1873] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:50.080433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.087929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbd2c9-be9b-499e-bb5e-c90973e09312 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.090588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b101aee0dfc54afbb9814914061a0e2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.093346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666722e3-8a52-4bb7-a2f2-b45376a8ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.093933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666722e3-8a52-4bb7-a2f2-b45376a8ea31 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.094212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666722e3-8a52-4bb7-a2f2-b45376a8ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.094486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666722e3-8a52-4bb7-a2f2-b45376a8ea31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.095104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666722e3-8a52-4bb7-a2f2-b45376a8ea31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.097046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666722e3-8a52-4bb7-a2f2-b45376a8ea31 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.098132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b101aee0dfc54afbb9814914061a0e2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.098808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b101aee0dfc54afbb9814914061a0e2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.106465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b101aee0dfc54afbb9814914061a0e2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.120926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b101aee0dfc54afbb9814914061a0e2c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.126086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 945/1874] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:50.127192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.130524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.130671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 930/1875] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:50 2022] DELETE /identity/v3/users/b101aee0dfc54afbb9814914061a0e2c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:50.130780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.130866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.130866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.142456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.149468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] MFA Rules not processed for user `bae100a3a8e94ca6b2800df52ab453f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.150676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.171117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.212610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed27f75-5098-4d98-bf6e-bd91d3284975 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.218326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.219162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.219448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.219721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] PATH_INFO: `/v3/projects/0bca7b0115544e1d86486b7b760c4524/users/e25776edcca04c648ee03d35f10c10c9/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.231990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 946/1876] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:50.232706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.234194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.236830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bca7b0115544e1d86486b7b760c4524, user_id=e25776edcca04c648ee03d35f10c10c9, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.240500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.248515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090516d-fe07-4b45-ae57-00f214c9df3f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.257284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 931/1877] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:50 2022] PUT /identity/v3/projects/0bca7b0115544e1d86486b7b760c4524/users/e25776edcca04c648ee03d35f10c10c9/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:50.258189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.260013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.337340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535f8fb-586d-4702-a62b-2d7b9e179c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.338030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535f8fb-586d-4702-a62b-2d7b9e179c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.338488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535f8fb-586d-4702-a62b-2d7b9e179c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.339284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535f8fb-586d-4702-a62b-2d7b9e179c71 admin admin] PATH_INFO: `/v3/projects/ecf00be381574f3086bf62a7be65cec8/users/8d4e5abdc1974a158b12a74f140e054d/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.341067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec178d95-6247-43a3-8b62-b5aefce50aac admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.345929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.346523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.346797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.347076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.347705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.349698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.356121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535f8fb-586d-4702-a62b-2d7b9e179c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf00be381574f3086bf62a7be65cec8, user_id=8d4e5abdc1974a158b12a74f140e054d, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.359025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535f8fb-586d-4702-a62b-2d7b9e179c71 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.379261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 932/1878] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:50.380133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.391052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 947/1879] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:50 2022] PUT /identity/v3/projects/ecf00be381574f3086bf62a7be65cec8/users/8d4e5abdc1974a158b12a74f140e054d/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:50.391052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.391052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.391052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.391887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.434051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] MFA Rules not processed for user `e25776edcca04c648ee03d35f10c10c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.435449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.509704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5003d-95d1-49ee-9902-ad9be0655988 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.515220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 948/1880] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:50.520458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.521834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.531392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.561001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.584887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.585620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.585962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.586241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.595983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.598238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.601385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6e87d-7254-4abd-a4f0-f9c1993e3cdb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.611067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 933/1881] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:50.611888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.613608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.614366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.614751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.615127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] PATH_INFO: `/v3/projects/1c43f2faabb842d4b37c8beb665c136e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.616456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c43f2faabb842d4b37c8beb665c136e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.618755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.622743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798726-7be9-494d-a862-ede4350be611 service placement] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.627606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dee78-cc8e-4319-ac19-8fe0c86e6154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.628220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dee78-cc8e-4319-ac19-8fe0c86e6154 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.628623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dee78-cc8e-4319-ac19-8fe0c86e6154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.628927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dee78-cc8e-4319-ac19-8fe0c86e6154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.629561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dee78-cc8e-4319-ac19-8fe0c86e6154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.634056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dee78-cc8e-4319-ac19-8fe0c86e6154 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.650600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 934/1882] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:50.650600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.650600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.650600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.674142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c43f2faabb842d4b37c8beb665c136e', 'request_id': 'req-31a13c1e-8265-467e-8cae-bf11b9d5ad00'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.678424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] MFA Rules not processed for user `8d4e5abdc1974a158b12a74f140e054d`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.679654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.680474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c43f2faabb842d4b37c8beb665c136e', 'request_id': 'req-31a13c1e-8265-467e-8cae-bf11b9d5ad00'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.689004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c43f2faabb842d4b37c8beb665c136e', 'request_id': 'req-31a13c1e-8265-467e-8cae-bf11b9d5ad00'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.701409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] The token cache is being invalidate because project 1c43f2faabb842d4b37c8beb665c136e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:50.701702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.702363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.703875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 949/1883] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:50 2022] DELETE /identity/v3/projects/1c43f2faabb842d4b37c8beb665c136e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:50.705516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.707555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.748378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 935/1884] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:50.749842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.751165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.773881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a13c1e-8265-467e-8cae-bf11b9d5ad00 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.778696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.779292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.779566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.779835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.780477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.782449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.812873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 950/1885] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:50.812873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.814449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.827530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263557e9-b8ff-48ed-94e4-3bc1427ffcbc None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.862924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.866959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.886364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05428c77-888f-44e1-b8bb-29fee23d154f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.890808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.891390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.891526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.891784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.892063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] PATH_INFO: `/v3/projects/ea54b0e8cb514df880179e9bb16e1805/users/b041ee86445141118b0d87484403fd69/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.892479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.892883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.893273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] PATH_INFO: `/v3/users/0e97d47baa8c4ee28cf48098885ada2d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.894704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e97d47baa8c4ee28cf48098885ada2d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.897701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.913408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea54b0e8cb514df880179e9bb16e1805, user_id=b041ee86445141118b0d87484403fd69, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.916236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.931823 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 936/1886] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:50 2022] PUT /identity/v3/projects/ea54b0e8cb514df880179e9bb16e1805/users/b041ee86445141118b0d87484403fd69/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:50.933107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.934681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.954924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e97d47baa8c4ee28cf48098885ada2d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.962872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e97d47baa8c4ee28cf48098885ada2d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.977513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e97d47baa8c4ee28cf48098885ada2d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.978320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e97d47baa8c4ee28cf48098885ada2d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.985727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e97d47baa8c4ee28cf48098885ada2d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:50.991693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 951/1887] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:50 2022] DELETE /identity/v3/users/0e97d47baa8c4ee28cf48098885ada2d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:50.993634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.995130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.005525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba773201-c584-4408-8fb7-6e675df54b97 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.072125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a17518-9c49-4616-a2b9-330a0798134a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.080730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.081505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.081790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.082582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.083233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.084726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.085438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.085789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.085903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.086183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.087026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.091011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.112423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 952/1888] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:51.116848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.118837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.134102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279cb2b-7978-436e-a637-702100d55308 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.140497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.141168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.141976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.141976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.142520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.144510 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.147474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 937/1889] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:51.150620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.162544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 953/1890] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:51.162544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.164339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.194533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f942ad-762e-4c2a-92b8-d7ad9d747cbc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.237450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.253036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cde1d3-d33b-4654-a8c4-9170df65819a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.265473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.271184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.271816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.272206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.272489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.273240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.274999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.283816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.284417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.284689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.285004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.285670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.286017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 938/1891] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:51.286890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.287628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.297761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456590b-a137-4d56-8bc0-211520c053fe admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.303296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.303888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.304163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.304429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] PATH_INFO: `/v3/projects/ea54b0e8cb514df880179e9bb16e1805/users/b041ee86445141118b0d87484403fd69/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.306340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea54b0e8cb514df880179e9bb16e1805, user_id=b041ee86445141118b0d87484403fd69, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.309631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.310130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 954/1892] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:51 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:51.311034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.313187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.320887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 939/1893] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:51 2022] PUT /identity/v3/projects/ea54b0e8cb514df880179e9bb16e1805/users/b041ee86445141118b0d87484403fd69/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:51.321716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.325974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160c1d-fd58-4c5b-a8cc-729750ebfca2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.391174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.391747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.392052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.392378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] PATH_INFO: `/v3/projects/90b16c7c657744739fceeb943704aadb` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.393710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b16c7c657744739fceeb943704aadb)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.395692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.397231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a99c-0269-43ad-9bfd-ce6fcfd364c4 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.402634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.403293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.403575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.403976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] PATH_INFO: `/v3/projects/2ef5a27b0cdf458a81323ec094f47d21/users/a5ecb36f97c94866a25ac85f807736eb/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.420647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef5a27b0cdf458a81323ec094f47d21, user_id=a5ecb36f97c94866a25ac85f807736eb, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.423305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.437193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 940/1894] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:51 2022] PUT /identity/v3/projects/2ef5a27b0cdf458a81323ec094f47d21/users/a5ecb36f97c94866a25ac85f807736eb/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:51.438162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.450858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b16c7c657744739fceeb943704aadb', 'request_id': 'req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:51.457692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b16c7c657744739fceeb943704aadb', 'request_id': 'req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:51.466173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b16c7c657744739fceeb943704aadb', 'request_id': 'req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:51.477517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] The token cache is being invalidate because project 90b16c7c657744739fceeb943704aadb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:51.477806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:51.478445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e6a8a7ac-cb29-4ee3-b51f-5ba6f51d2a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:51.480629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 955/1895] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:51 2022] DELETE /identity/v3/projects/90b16c7c657744739fceeb943704aadb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:51.481855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4805927-fd6a-418e-8599-b0e92e73cb63 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.482130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4805927-fd6a-418e-8599-b0e92e73cb63 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.482414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4805927-fd6a-418e-8599-b0e92e73cb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.513862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d2204-918b-4005-a422-4a53a15e8b71 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.515573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f4805927-fd6a-418e-8599-b0e92e73cb63 None None] MFA Rules not processed for user `b041ee86445141118b0d87484403fd69`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:51.518130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f4805927-fd6a-418e-8599-b0e92e73cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.578700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.583342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.584077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.584420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.584827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.585542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.587974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.598898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 941/1896] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:51.598898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.603620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.611058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 956/1897] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:51.611058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.611058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.611058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.611947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.618505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896d79a-dfae-4b0d-9e09-b5f6b779445e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.625376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.625986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.626258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.626525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.627161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.628756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.643135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 942/1898] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:51.643135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.649958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.654473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:51.655676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.666183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 957/1899] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:51.667607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.671187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327abf36-218e-47f5-8882-592bcc838a8b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.673938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.693233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.694296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.694799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.695144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] PATH_INFO: `/v3/projects/2a2d0f7a8a364beb9fcb2f9ab355a2e0/users/5194d1f6fc07485baa9e9e29a4fad504/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.712468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2d0f7a8a364beb9fcb2f9ab355a2e0, user_id=5194d1f6fc07485baa9e9e29a4fad504, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.715912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.736854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 943/1900] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:51 2022] PUT /identity/v3/projects/2a2d0f7a8a364beb9fcb2f9ab355a2e0/users/5194d1f6fc07485baa9e9e29a4fad504/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:51.737761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.741345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe10122-c763-43ec-8acd-bbce4dae8453 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.805910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9041110-406a-447a-9711-857a65c8a13e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.811331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f78ef-8c5d-4e95-add9-3505ba38d7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.812052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f78ef-8c5d-4e95-add9-3505ba38d7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.812397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f78ef-8c5d-4e95-add9-3505ba38d7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.812736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f78ef-8c5d-4e95-add9-3505ba38d7fd admin admin] PATH_INFO: `/v3/projects/2ef5a27b0cdf458a81323ec094f47d21/users/a5ecb36f97c94866a25ac85f807736eb/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.814735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f78ef-8c5d-4e95-add9-3505ba38d7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef5a27b0cdf458a81323ec094f47d21, user_id=a5ecb36f97c94866a25ac85f807736eb, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.817549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f78ef-8c5d-4e95-add9-3505ba38d7fd admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.820598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.821267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.821548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.821822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.828036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 944/1901] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:51 2022] PUT /identity/v3/projects/2ef5a27b0cdf458a81323ec094f47d21/users/a5ecb36f97c94866a25ac85f807736eb/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:51.829255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e79165-fa93-43db-b8b5-8c243bdfc8a7 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.829538 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e79165-fa93-43db-b8b5-8c243bdfc8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.829812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e79165-fa93-43db-b8b5-8c243bdfc8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.848756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f3e79165-fa93-43db-b8b5-8c243bdfc8a7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:51.849981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f3e79165-fa93-43db-b8b5-8c243bdfc8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.886499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.888033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.898216 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 958/1902] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:51.899054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.913136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06ef59-b946-4019-b34c-c1bf516342d1 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.917739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.918341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.922396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.922524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 945/1903] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:51.922936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.922936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.923069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c261ff8-f367-497c-a4ed-7a746b4a1208 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.923069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c261ff8-f367-497c-a4ed-7a746b4a1208 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.923069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c261ff8-f367-497c-a4ed-7a746b4a1208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.923271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.933504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 959/1904] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:51.934466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.945627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db57b7-936d-47d8-8993-5d8f6c96d210 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.955425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.956036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.956154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-4c261ff8-f367-497c-a4ed-7a746b4a1208 None None] MFA Rules not processed for user `a5ecb36f97c94866a25ac85f807736eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:51.956311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.956579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.957256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:51.957406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4c261ff8-f367-497c-a4ed-7a746b4a1208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.957561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.959304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.972808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 960/1905] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:51.973663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.988002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04d287-ad2b-4de8-abfc-d4ef7d22f2f2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.993035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.993693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.994295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.994403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] PATH_INFO: `/v3/projects/2a2d0f7a8a364beb9fcb2f9ab355a2e0/users/5194d1f6fc07485baa9e9e29a4fad504/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.997152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2d0f7a8a364beb9fcb2f9ab355a2e0, user_id=5194d1f6fc07485baa9e9e29a4fad504, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.000114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.013623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 961/1906] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:51 2022] PUT /identity/v3/projects/2a2d0f7a8a364beb9fcb2f9ab355a2e0/users/5194d1f6fc07485baa9e9e29a4fad504/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:52.015023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.023493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.042933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.048474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 946/1907] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:52.049855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.050237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.050606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.080981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] MFA Rules not processed for user `5194d1f6fc07485baa9e9e29a4fad504`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:52.082432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:52.105989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabec9b1-a778-4b36-ad26-01a8fea655b1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.110680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.111285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.111565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.111862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.112494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.114453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.144833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 962/1908] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:52 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:52.149168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.163851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 947/1909] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:52.163851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.174500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429b850-6955-48ad-8be5-9480ac0c0f09 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.221678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34bf5b-bae5-4cb9-ae1d-0bc3d627a108 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.226357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.227299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.227762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.228143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.248631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.250417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.250942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.256684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.256684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.257115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.299496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 948/1910] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:52 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:52.300418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.302044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:52.313855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.315490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.317006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ccedb-73a8-4e05-8758-c73191419c87 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.321913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.322263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.322581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.323184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.337382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 963/1911] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:52.338363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.361892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc4cd0-3c33-4a66-a495-3131da359d94 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.375246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.376703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.377009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.377282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.377924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.381441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.402992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.406697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.409013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.410682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 964/1912] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:52.411673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.424994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 949/1913] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:52.436907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29eb59-500d-4ff2-b61f-1fc418027d78 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.458137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.467123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.467755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.468028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.468304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.468986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.471065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.491021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 965/1914] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:52.496442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.507502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4299a6-a7e4-487b-b605-648e289f44fa admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.513398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40c427-be2a-4f48-ad1e-6e2edbde2e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.514599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c427-be2a-4f48-ad1e-6e2edbde2e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.514880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c427-be2a-4f48-ad1e-6e2edbde2e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.515329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c427-be2a-4f48-ad1e-6e2edbde2e23 admin admin] PATH_INFO: `/v3/projects/01f101a51c724982ab83002f89d59829/users/8972d3c7ac5d4fcc8c4e937a5e12db90/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.538810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c427-be2a-4f48-ad1e-6e2edbde2e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f101a51c724982ab83002f89d59829, user_id=8972d3c7ac5d4fcc8c4e937a5e12db90, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.543091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c427-be2a-4f48-ad1e-6e2edbde2e23 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.560335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.571842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 966/1915] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:52 2022] PUT /identity/v3/projects/01f101a51c724982ab83002f89d59829/users/8972d3c7ac5d4fcc8c4e937a5e12db90/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:52.588010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac3bf0-9639-4631-b65c-d026619b2316 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.588199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.588368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.588679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.628956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] MFA Rules not processed for user `8972d3c7ac5d4fcc8c4e937a5e12db90`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:52.630418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:52.685613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b041ee86445141118b0d87484403fd69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea54b0e8cb514df880179e9bb16e1805', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.686552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.686842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.687128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.688235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:52.688559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.690242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.708149 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 950/1916] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:52.709751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.725644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 967/1917] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:52.726608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.750077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100434cd-6f68-4038-84ad-60689894ef4d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.761910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b041ee86445141118b0d87484403fd69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea54b0e8cb514df880179e9bb16e1805', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.762516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.762799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.763083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.763718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.765720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.823029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 968/1918] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:52 2022] POST /identity/v3/projects => generated 354 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:52.823972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.858023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262fde-bd91-4a97-9fc1-7c95850d94b9 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.922815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ce651-d1d2-4169-9bde-db11dfa10d9c tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.928885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.929512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.929794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.930072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.930735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.933093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.933886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.951078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.951723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.951998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.952275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] PATH_INFO: `/v3/users/8d4e5abdc1974a158b12a74f140e054d` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.953462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4e5abdc1974a158b12a74f140e054d)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.955748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.982575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 969/1919] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:52 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:52.983867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.045183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4e5abdc1974a158b12a74f140e054d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.061542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4e5abdc1974a158b12a74f140e054d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.065715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4e5abdc1974a158b12a74f140e054d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.072031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.076454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4e5abdc1974a158b12a74f140e054d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.094300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c583e-8216-4d77-9707-f9e98be5efb6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.100156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b041ee86445141118b0d87484403fd69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea54b0e8cb514df880179e9bb16e1805', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.101691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.102080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.102373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.103301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:53.103612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.106608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4e5abdc1974a158b12a74f140e054d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.107340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fda630-32e1-4239-ac57-a4af17e4d0c4 tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.121986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 951/1920] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:52 2022] DELETE /identity/v3/users/8d4e5abdc1974a158b12a74f140e054d => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:53.122941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.127001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 970/1921] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:53.131463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.131769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.132273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.139163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.163320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:53.164907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.239239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a444caf-5d39-4b2c-ba08-88829895a124 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.245047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.245770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.246053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.246330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.246981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.249370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.256534 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 971/1922] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:53.258160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.260151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.283927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 952/1923] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:53 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:53.285020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.297246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f54b2-3858-4ad9-83cb-8e75a72edd65 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.302546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.303161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.303439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.303773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.304488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.306096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.322083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 953/1924] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:53.323541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.335454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2628-674f-46b3-ae39-a4c7e1b8de7a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.341327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.342138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.342424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.342706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] PATH_INFO: `/v3/projects/228edafff7b54a00a734dfd387294045/users/497a903d6f34449c87f0284ed5c84454/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.361774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228edafff7b54a00a734dfd387294045, user_id=497a903d6f34449c87f0284ed5c84454, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.364845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.365177 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9b0f3-1bde-4073-87c4-9246f75acd01 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.374739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b041ee86445141118b0d87484403fd69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea54b0e8cb514df880179e9bb16e1805', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.376137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.377718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.378193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] PATH_INFO: `/v3/projects/6907f98085e04269b53a3f6f46c7f1e2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.380262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6907f98085e04269b53a3f6f46c7f1e2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.384309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.402552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 954/1925] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:53 2022] PUT /identity/v3/projects/228edafff7b54a00a734dfd387294045/users/497a903d6f34449c87f0284ed5c84454/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:53.403734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.448090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.515501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6907f98085e04269b53a3f6f46c7f1e2', 'request_id': 'req-61cf8a85-228a-421c-94fd-b2b656107a6d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.534743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6907f98085e04269b53a3f6f46c7f1e2', 'request_id': 'req-61cf8a85-228a-421c-94fd-b2b656107a6d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.560668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6907f98085e04269b53a3f6f46c7f1e2', 'request_id': 'req-61cf8a85-228a-421c-94fd-b2b656107a6d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.567455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847447f-5b1f-48f6-a3d0-b48f9732c4f4 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.585281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] The token cache is being invalidate because project 6907f98085e04269b53a3f6f46c7f1e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:53.586676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.588470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:53.591243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 972/1926] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:53 2022] DELETE /identity/v3/projects/6907f98085e04269b53a3f6f46c7f1e2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:53.592062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.593483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.655030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.720487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.724616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.726711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.726972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.728485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.731415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.744102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf8a85-228a-421c-94fd-b2b656107a6d tempest-VolumeQuotaClassesTest-29137735 tempest-VolumeQuotaClassesTest-29137735-project-admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.750426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.751286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.752442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.752888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.754594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 955/1927] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:53.759776 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.762574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.767826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.781070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.782848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.790781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a16a9-15a8-41d6-8699-659190a44bb7 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.797102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72b909-98bb-4ab2-ac34-36fec90810af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.797102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72b909-98bb-4ab2-ac34-36fec90810af admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.797102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72b909-98bb-4ab2-ac34-36fec90810af admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.797530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72b909-98bb-4ab2-ac34-36fec90810af admin admin] PATH_INFO: `/v3/projects/228edafff7b54a00a734dfd387294045/users/497a903d6f34449c87f0284ed5c84454/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.799730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72b909-98bb-4ab2-ac34-36fec90810af admin admin] RBAC: Authorizing `identity:create_grant(project_id=228edafff7b54a00a734dfd387294045, user_id=497a903d6f34449c87f0284ed5c84454, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.802384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72b909-98bb-4ab2-ac34-36fec90810af admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.832052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 956/1928] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:53 2022] PUT /identity/v3/projects/228edafff7b54a00a734dfd387294045/users/497a903d6f34449c87f0284ed5c84454/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:53.837395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.837395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.837395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.841323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 973/1929] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:53.871640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] MFA Rules not processed for user `497a903d6f34449c87f0284ed5c84454`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:53.873336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.955194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.957843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:53.978055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf47e1e-6f5b-4c49-960b-78ee958add44 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.045484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 957/1930] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:54.061175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.064186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.070019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.070783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.071209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.071639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] PATH_INFO: `/v3/users/d833e588bc28485db7e1af8c96753cb6` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.073930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d833e588bc28485db7e1af8c96753cb6)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.076395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.118411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.167121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662c9f1-188b-4993-a2c6-a273b581fc74 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.167329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd833e588bc28485db7e1af8c96753cb6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.181417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4222f0-bbed-4327-a53e-38deef5770b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.182070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4222f0-bbed-4327-a53e-38deef5770b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.182895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4222f0-bbed-4327-a53e-38deef5770b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.185604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4222f0-bbed-4327-a53e-38deef5770b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.194576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd833e588bc28485db7e1af8c96753cb6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.211044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4222f0-bbed-4327-a53e-38deef5770b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.215650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4222f0-bbed-4327-a53e-38deef5770b3 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.229083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd833e588bc28485db7e1af8c96753cb6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.233715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd833e588bc28485db7e1af8c96753cb6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.260498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd833e588bc28485db7e1af8c96753cb6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.263545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 958/1931] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:54.272249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 974/1932] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:53 2022] DELETE /identity/v3/users/d833e588bc28485db7e1af8c96753cb6 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:54.273609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.274006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.274289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.305939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:54.310939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.406879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.408595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.532847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 959/1933] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:54.535426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.537771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.559072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b122b-9b7d-44d5-b4ee-eb572874f08b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.565861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.566866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.567951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.568343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] PATH_INFO: `/v3/projects/ecf00be381574f3086bf62a7be65cec8` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.569758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf00be381574f3086bf62a7be65cec8)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.572332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.578946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.600258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecddcd4-6703-48d3-8b85-e504832b8695 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.607314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.609207 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.609770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.610184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.612700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.653558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf00be381574f3086bf62a7be65cec8', 'request_id': 'req-8566aa15-e77c-4776-a4ed-5095300d6bcf'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.663365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf00be381574f3086bf62a7be65cec8', 'request_id': 'req-8566aa15-e77c-4776-a4ed-5095300d6bcf'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.674082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf00be381574f3086bf62a7be65cec8', 'request_id': 'req-8566aa15-e77c-4776-a4ed-5095300d6bcf'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.689187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] The token cache is being invalidate because project ecf00be381574f3086bf62a7be65cec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:54.690517 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.691338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:54.697192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 960/1934] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:54 2022] DELETE /identity/v3/projects/ecf00be381574f3086bf62a7be65cec8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:54.699451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.704423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.749694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.752903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.788252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 975/1935] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:54.790843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.813944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530a0-b288-4aa9-bf04-34c8c3f1fd16 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.891119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566aa15-e77c-4776-a4ed-5095300d6bcf admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.906732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.908805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.909110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.910663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.915803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.954269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.984621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.986331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.986723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.989274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] PATH_INFO: `/v3/users/5194d1f6fc07485baa9e9e29a4fad504` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.992104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5194d1f6fc07485baa9e9e29a4fad504)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.995494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.057117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5194d1f6fc07485baa9e9e29a4fad504'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:55.073046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5194d1f6fc07485baa9e9e29a4fad504'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:55.126654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5194d1f6fc07485baa9e9e29a4fad504'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:55.127594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5194d1f6fc07485baa9e9e29a4fad504'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:55.132600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.134105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.148400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef69ddd4-dbe8-4e9b-a61c-d96b3db540c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5194d1f6fc07485baa9e9e29a4fad504'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:55.168431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 961/1936] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:55.172430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 976/1937] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:54 2022] DELETE /identity/v3/users/5194d1f6fc07485baa9e9e29a4fad504 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:55.172984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.172984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:55.217416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe067d-062d-409d-8617-b74b58ca2981 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.217416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe067d-062d-409d-8617-b74b58ca2981 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.217741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe067d-062d-409d-8617-b74b58ca2981 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.219634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 977/1938] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:55.273991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.276247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.277384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.281714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f32f3-69c2-4340-85d9-a2382333f228 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.365158 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:55.371134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:55.407981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.425745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.426621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.427079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.427708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.438939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:55.587167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.589939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.613776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 978/1939] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:30:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:55.628290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 962/1940] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:55.640902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.642521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:55.681094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b84d-1ac7-4451-b91c-97f1e0f7dd72 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.691714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.693051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.693284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.693713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.721081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.722858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.764733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 979/1941] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:55.831560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.834574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:55.938458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.024379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60eafe-bab2-4814-9fd4-cf978de011cb service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.031581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.032220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.032519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.032793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] PATH_INFO: `/v3/projects/a1557b629ebc4626a836ba83ff86c453` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.033928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1557b629ebc4626a836ba83ff86c453)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.036382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.056123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.059027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:56.078483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eebf11-7e54-41c6-bb7d-fea3a9a1d4ff service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.085512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.086123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.086416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.086656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] PATH_INFO: `/v3/projects/2a2d0f7a8a364beb9fcb2f9ab355a2e0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.087789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2d0f7a8a364beb9fcb2f9ab355a2e0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.090298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.113845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1557b629ebc4626a836ba83ff86c453', 'request_id': 'req-3f4174b0-f176-490a-9dd8-75c4be05f122'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.157271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1557b629ebc4626a836ba83ff86c453', 'request_id': 'req-3f4174b0-f176-490a-9dd8-75c4be05f122'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.164571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1557b629ebc4626a836ba83ff86c453', 'request_id': 'req-3f4174b0-f176-490a-9dd8-75c4be05f122'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.181521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] The token cache is being invalidate because project a1557b629ebc4626a836ba83ff86c453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:56.181632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.183403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.198439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 963/1942] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:55 2022] DELETE /identity/v3/projects/a1557b629ebc4626a836ba83ff86c453 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:56.199269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2d0f7a8a364beb9fcb2f9ab355a2e0', 'request_id': 'req-e3bdfb23-c52f-4b9c-9074-22896b20842c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.200673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.202300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:56.207568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2d0f7a8a364beb9fcb2f9ab355a2e0', 'request_id': 'req-e3bdfb23-c52f-4b9c-9074-22896b20842c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.219915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2d0f7a8a364beb9fcb2f9ab355a2e0', 'request_id': 'req-e3bdfb23-c52f-4b9c-9074-22896b20842c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.242334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] The token cache is being invalidate because project 2a2d0f7a8a364beb9fcb2f9ab355a2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:56.242897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.244556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e3bdfb23-c52f-4b9c-9074-22896b20842c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.248839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 980/1943] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:56 2022] DELETE /identity/v3/projects/2a2d0f7a8a364beb9fcb2f9ab355a2e0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:56.253013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe4149-4adc-4a48-8de8-ccaa21016a0c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.253582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe4149-4adc-4a48-8de8-ccaa21016a0c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.254042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe4149-4adc-4a48-8de8-ccaa21016a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.300962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b6fe4149-4adc-4a48-8de8-ccaa21016a0c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:56.302482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b6fe4149-4adc-4a48-8de8-ccaa21016a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:56.319637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4174b0-f176-490a-9dd8-75c4be05f122 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.403760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.408637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 981/1944] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:56.417838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.418487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.418605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.418925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.419640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.419867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] PATH_INFO: `/v3/users/b041ee86445141118b0d87484403fd69` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.420981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.421592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b041ee86445141118b0d87484403fd69)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.423890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.448052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:56.449813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:56.459794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 982/1945] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:56.463635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.488296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dd777-9f69-4efc-bec9-b315661d1c30 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.499907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.500741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.501031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.501299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.501943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.502581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.503852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.506636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.525566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 983/1946] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.527650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.530214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b041ee86445141118b0d87484403fd69'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.541659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b041ee86445141118b0d87484403fd69'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.542692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b041ee86445141118b0d87484403fd69'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.543595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c63659-9d64-4de2-810e-1199a259f2d4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.552393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.554859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.555416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.555569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.555651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b041ee86445141118b0d87484403fd69'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.556230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.560576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.572137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b041ee86445141118b0d87484403fd69'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:56.584161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 964/1947] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:56 2022] DELETE /identity/v3/users/b041ee86445141118b0d87484403fd69 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:56.612482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 984/1948] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:56 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:56.616588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.618418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:56.741449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81438-8cda-480c-95b0-698caaa0b47b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.851905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.853605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.853605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.854109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.855385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.858045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.909409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 965/1949] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:56 2022] POST /identity/v3/users => generated 286 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:56.915541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.960867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381afd58-0159-408e-9b6c-711d0222e2b1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.994893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.996402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.998925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.999740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.002062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.006369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.035765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.047346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 985/1950] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:57.051484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebabae-89a5-4db4-a2b4-352b01962eec admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.068217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20d08e-0c14-4981-a5bb-725bad4e4522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.078835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20d08e-0c14-4981-a5bb-725bad4e4522 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.086105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20d08e-0c14-4981-a5bb-725bad4e4522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.088654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20d08e-0c14-4981-a5bb-725bad4e4522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.089714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20d08e-0c14-4981-a5bb-725bad4e4522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.094266 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20d08e-0c14-4981-a5bb-725bad4e4522 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.113204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 966/1951] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:57.116602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.128311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340dea-efdd-4268-b544-a00be35bcc04 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.135339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.135549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.135766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.137624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] PATH_INFO: `/v3/projects/1a571e23af424b0ea5e4957b2a66617c/users/503240c021c141fd9ebb49211e16a63d/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.162348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a571e23af424b0ea5e4957b2a66617c, user_id=503240c021c141fd9ebb49211e16a63d, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.166472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.197200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 986/1952] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:57 2022] PUT /identity/v3/projects/1a571e23af424b0ea5e4957b2a66617c/users/503240c021c141fd9ebb49211e16a63d/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:57.201821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.202760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.203355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.212246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.282971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] MFA Rules not processed for user `503240c021c141fd9ebb49211e16a63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:57.284456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.347760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.350814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.499774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 967/1953] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:57.551123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b730c82-bdd6-4574-8354-6ef72a9ea81f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.560017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.561755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.569984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.572535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.572908 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.574458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] PATH_INFO: `/v3/projects/ea54b0e8cb514df880179e9bb16e1805` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.578279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea54b0e8cb514df880179e9bb16e1805)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.586168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.695470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c557f-4ee9-4006-934c-6cca52e4f94b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.723546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea54b0e8cb514df880179e9bb16e1805', 'request_id': 'req-08efd19e-6a8a-43a0-a4ca-2866f645c12a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:57.742847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea54b0e8cb514df880179e9bb16e1805', 'request_id': 'req-08efd19e-6a8a-43a0-a4ca-2866f645c12a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:57.753279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea54b0e8cb514df880179e9bb16e1805', 'request_id': 'req-08efd19e-6a8a-43a0-a4ca-2866f645c12a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:57.767787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] The token cache is being invalidate because project ea54b0e8cb514df880179e9bb16e1805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:57.768099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:57.768844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-08efd19e-6a8a-43a0-a4ca-2866f645c12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:57.771232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 987/1954] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:57 2022] DELETE /identity/v3/projects/ea54b0e8cb514df880179e9bb16e1805 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:57.775500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.775783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.776059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.779691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.780609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.781243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.781368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.783286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.801811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:57.803171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.846476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.876944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.878476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e518-0f1e-44fc-ac8d-9cee2b840f5e service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.890127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 968/1955] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:57.897017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 988/1956] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:57.905703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.905703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.905703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.913210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.918291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.919696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.930837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:57.932281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.932388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7ad70-b436-4a26-a27e-219569cc51a6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.938440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.939076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.939355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.939773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] PATH_INFO: `/v3/users/503240c021c141fd9ebb49211e16a63d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.941199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503240c021c141fd9ebb49211e16a63d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.944817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 969/1957] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:57.944951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.947477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.959020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f1417-64e1-4492-a587-be4c8c2e1e87 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.967435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.968121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.968405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.968695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.969666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:57.969998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.972003 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.990654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 970/1958] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:57.992054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.998796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503240c021c141fd9ebb49211e16a63d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:58.005355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0c041-69c5-4ff6-9042-70c1156fcbc3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.006537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503240c021c141fd9ebb49211e16a63d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:58.012244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.013152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.013635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.013816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.014674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.017613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.021838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503240c021c141fd9ebb49211e16a63d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:58.022494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503240c021c141fd9ebb49211e16a63d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:58.030537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1dec61aa-d20d-4766-b097-6773a6447c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503240c021c141fd9ebb49211e16a63d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:58.036610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 989/1959] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:57 2022] DELETE /identity/v3/users/503240c021c141fd9ebb49211e16a63d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:58.041749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.042088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.043452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.048813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 971/1960] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:57 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:58.052998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.054897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:58.063843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:58.065067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:58.130004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4229e5e-f370-4916-87d6-548a7f6b51bc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.134488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.135105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.135377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.135653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.137443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 990/1961] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:58.137563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.139564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.151654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.153835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:58.173014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 972/1962] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:58 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:30:58.173014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.176379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62c0f2-36d5-4280-9839-05dabdb63507 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.181447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.182053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.182329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.183199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.183935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.185481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.197618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 973/1963] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:30:58.202630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.208674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:58.223501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6e7b3-92c4-4b35-a64e-07ede7bc4aa9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.227195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6e39b-ead4-4d7f-ad7c-4f177b9e8b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.227330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf543364-0969-49eb-b79a-080843b89f3c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.227827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6e39b-ead4-4d7f-ad7c-4f177b9e8b48 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.228139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6e39b-ead4-4d7f-ad7c-4f177b9e8b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.228481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6e39b-ead4-4d7f-ad7c-4f177b9e8b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.234600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.235245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.235653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.235839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.236512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.238229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.239340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6e39b-ead4-4d7f-ad7c-4f177b9e8b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.240861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6e39b-ead4-4d7f-ad7c-4f177b9e8b48 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.251404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 974/1964] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:30:58.253838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 991/1965] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:58.255420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.268732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031da01-c854-453e-bef1-385769c74479 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.274164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.280674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.280674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.280674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] PATH_INFO: `/v3/projects/4d10a6d4c59c4bd4b5f49fe0fec3772c/users/a2ce922a4b7e483ea253a03a0072eb9d/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.298037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d10a6d4c59c4bd4b5f49fe0fec3772c, user_id=a2ce922a4b7e483ea253a03a0072eb9d, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.302166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.324228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 975/1966] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:58 2022] PUT /identity/v3/projects/4d10a6d4c59c4bd4b5f49fe0fec3772c/users/a2ce922a4b7e483ea253a03a0072eb9d/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:58.336677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.337038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.337294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.428380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] MFA Rules not processed for user `a2ce922a4b7e483ea253a03a0072eb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:58.429778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:58.509942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 992/1967] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:58.516406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.622832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf19d-3efc-405c-a527-390516cdb748 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.636719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.637715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.638097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.638473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.639346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.641431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.656386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:58.680611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 976/1968] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:58 2022] POST /identity/v3/projects => generated 399 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:58.684408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.704426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bbe85-57d5-481f-a056-33a55315b13c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.811226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.813376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:58.862280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.862919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.863198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.866325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.867024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.873317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.949467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 993/1969] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:58 2022] POST /identity/v3/users => generated 306 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:58.952269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.972417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b530-46f4-49c6-b03f-91220dd7ceea admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.977549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.978326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.978603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.978878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.979536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.982349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.996823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 994/1970] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:59.000024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.042108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe44f7-85e6-43cc-951d-226251f91cf4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.056094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.057911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.059051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.060573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] PATH_INFO: `/v3/projects/2f0de9c4dcc0434aa496d79275fe72af/users/f0d8e50508be4e45b53e809b31762647/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.072803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae0440-2e77-4e8c-a6ba-9bcaee7437ea admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.088850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.090926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.091198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.091475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] PATH_INFO: `/v3/users/27ff216440a948d5b3a97bd942eb7398` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.093486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ff216440a948d5b3a97bd942eb7398)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.095924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.109369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0de9c4dcc0434aa496d79275fe72af, user_id=f0d8e50508be4e45b53e809b31762647, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.112856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.173381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 995/1971] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:58 2022] PUT /identity/v3/projects/2f0de9c4dcc0434aa496d79275fe72af/users/f0d8e50508be4e45b53e809b31762647/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:59.176494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.180366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.235325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.295159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ff216440a948d5b3a97bd942eb7398'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.315578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801df64-362f-4eec-9d66-9672b852d27c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.321492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ff216440a948d5b3a97bd942eb7398'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.322789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ff216440a948d5b3a97bd942eb7398'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.324012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.324923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.325443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.325603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] PATH_INFO: `/v3/projects/1a571e23af424b0ea5e4957b2a66617c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.326861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a571e23af424b0ea5e4957b2a66617c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.330049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.359025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ff216440a948d5b3a97bd942eb7398'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.395157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ff216440a948d5b3a97bd942eb7398'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.404376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a571e23af424b0ea5e4957b2a66617c', 'request_id': 'req-1101820f-2825-4b48-be43-d5740f8ab543'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.411025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a571e23af424b0ea5e4957b2a66617c', 'request_id': 'req-1101820f-2825-4b48-be43-d5740f8ab543'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.419339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 977/1972] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:30:58 2022] DELETE /identity/v3/users/27ff216440a948d5b3a97bd942eb7398 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:59.423623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.425888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.430303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a571e23af424b0ea5e4957b2a66617c', 'request_id': 'req-1101820f-2825-4b48-be43-d5740f8ab543'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.460706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] The token cache is being invalidate because project 1a571e23af424b0ea5e4957b2a66617c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:30:59.461026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.462940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1101820f-2825-4b48-be43-d5740f8ab543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:30:59.466388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 996/1973] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:30:59 2022] DELETE /identity/v3/projects/1a571e23af424b0ea5e4957b2a66617c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:59.467380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.467662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.467935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.490307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.491782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.522031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa60772-1aaf-409c-8aa3-4502532eef5a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.529190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d496-5082-44f2-bb29-eb087e580a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.529952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d496-5082-44f2-bb29-eb087e580a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.530248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d496-5082-44f2-bb29-eb087e580a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.530521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d496-5082-44f2-bb29-eb087e580a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.532401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8d496-5082-44f2-bb29-eb087e580a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.534077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8d496-5082-44f2-bb29-eb087e580a47 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.546215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 978/1974] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:30:59.547475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.547761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.548036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.560234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.572434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 997/1975] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:59.573388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.575073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.577697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.579369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.587447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5b361-d8a1-4be7-a48d-976e081c019d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.588073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 979/1976] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:59.588932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.591242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.594331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c1789-a40b-49f8-a4ce-4fec6410c7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.595179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c1789-a40b-49f8-a4ce-4fec6410c7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.595932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c1789-a40b-49f8-a4ce-4fec6410c7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.596358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c1789-a40b-49f8-a4ce-4fec6410c7f7 admin admin] PATH_INFO: `/v3/projects/2f0de9c4dcc0434aa496d79275fe72af/users/f0d8e50508be4e45b53e809b31762647/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.599312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c1789-a40b-49f8-a4ce-4fec6410c7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0de9c4dcc0434aa496d79275fe72af, user_id=f0d8e50508be4e45b53e809b31762647, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.602082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c1789-a40b-49f8-a4ce-4fec6410c7f7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.613502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 998/1977] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:30:59 2022] PUT /identity/v3/projects/2f0de9c4dcc0434aa496d79275fe72af/users/f0d8e50508be4e45b53e809b31762647/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:59.614599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c4780-d496-446a-8483-4f024a3e85ce None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.614887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c4780-d496-446a-8483-4f024a3e85ce None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.615150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c4780-d496-446a-8483-4f024a3e85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.645513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-783c4780-d496-446a-8483-4f024a3e85ce None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.646742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-783c4780-d496-446a-8483-4f024a3e85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.679883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a775d-23ab-4ebb-8dcb-883fdde9beed None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.702768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.732837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 999/1978] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:59.734474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.734934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.735366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.767304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.768342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.768681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.769173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.781618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.783404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.796122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 980/1979] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:59.797049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.810468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02f0e-af5f-44d8-8925-37115880db2b service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.813201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] MFA Rules not processed for user `f0d8e50508be4e45b53e809b31762647`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.814704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.816420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.817267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.817650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.818851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.819599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:59.820057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.821889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.837556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 981/1980] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:59.844567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.858395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0948c2-37a8-416e-ba46-1cf7efcb4ba9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.863925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.864680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.865301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.865603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.866413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.868713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.924279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 982/1981] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:59.928580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.970620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1000/1982] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:59.982604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.984045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.031630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac573396-8ecf-43b1-a8b4-575fdafde8ba admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.037131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.038109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.038222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.038411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.039090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.041075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.077617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82984bc-03dd-4096-bb6a-4a1f94622970 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.079425 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 983/1983] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:00.083484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.095056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009add-9ae0-4dd9-8937-8326a4781e58 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.101424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.102221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.102497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.102765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.103585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.105696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.111766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.118638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 984/1984] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:00.122363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.132747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917044b4-c55d-4afe-86b0-b6d258d4a526 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.138008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.138674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.138957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.139230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.139985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.141735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.143663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e0e2-aae6-47a0-9b43-5bd588c325f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.144845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e0e2-aae6-47a0-9b43-5bd588c325f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.145132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e0e2-aae6-47a0-9b43-5bd588c325f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.145429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e0e2-aae6-47a0-9b43-5bd588c325f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.158644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 985/1985] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:00.160317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.176344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd15a4-b7dd-493c-a318-8051d1820b4e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.181400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.182052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.182411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.182731 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] PATH_INFO: `/v3/projects/0a371d5a13d54a01b879645b26a1ebe9/users/69e66e65f5eb4816a5e8fd5381a9e2a8/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.203375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a371d5a13d54a01b879645b26a1ebe9, user_id=69e66e65f5eb4816a5e8fd5381a9e2a8, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.206225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.206800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.229263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 986/1986] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:00 2022] PUT /identity/v3/projects/0a371d5a13d54a01b879645b26a1ebe9/users/69e66e65f5eb4816a5e8fd5381a9e2a8/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:00.231094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.233601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.236802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e0e2-aae6-47a0-9b43-5bd588c325f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.238486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e0e2-aae6-47a0-9b43-5bd588c325f5 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.250666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1001/1987] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:00.252222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.252492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.252799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.295268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] MFA Rules not processed for user `69e66e65f5eb4816a5e8fd5381a9e2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:00.296882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.383158 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee328c3-851b-4f8d-9bc1-b9ac6865e108 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.397217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.398225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.398729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.399058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] PATH_INFO: `/v3/projects/abaa6b53cb4145a697de678d0debcc41` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.401112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abaa6b53cb4145a697de678d0debcc41)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.403888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.455612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1002/1988] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:00.455612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.459034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.471520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abaa6b53cb4145a697de678d0debcc41', 'request_id': 'req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.486038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abaa6b53cb4145a697de678d0debcc41', 'request_id': 'req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.496998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abaa6b53cb4145a697de678d0debcc41', 'request_id': 'req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.549619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] The token cache is being invalidate because project abaa6b53cb4145a697de678d0debcc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:00.549924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.562558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.565918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 987/1989] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:00 2022] DELETE /identity/v3/projects/abaa6b53cb4145a697de678d0debcc41 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:00.571357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.572979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.638824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107ef18-3463-4c96-ac94-9b5e3cf4e952 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.669174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.678400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc896e-45fb-47c8-aa88-60ca63dd4ea4 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.691912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.693327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.695096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.695603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] PATH_INFO: `/v3/users/7cf1dca5e27f4136bb3a74a1ba070371` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.697254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf1dca5e27f4136bb3a74a1ba070371)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.700375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.716351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.717128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.717476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.717853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.720278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.786030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf1dca5e27f4136bb3a74a1ba070371'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.791673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.799854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf1dca5e27f4136bb3a74a1ba070371'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.801012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf1dca5e27f4136bb3a74a1ba070371'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.810480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf1dca5e27f4136bb3a74a1ba070371'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.815343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.816873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.824890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf1dca5e27f4136bb3a74a1ba070371'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:00.827264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1003/1990] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:00.831375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 988/1991] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:00 2022] DELETE /identity/v3/users/7cf1dca5e27f4136bb3a74a1ba070371 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:00.837906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.840126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.851236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6f696-bb08-47f0-b789-5013f4f84ce1 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.920656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.921544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.921977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.922433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.924722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:01.001729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.003693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.015765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1004/1992] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:01.504060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:01.506446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:01.650537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5318e98-3e62-49ff-aefb-1d856ef91c37 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.684710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.688245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.688541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.688862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] PATH_INFO: `/v3/projects/ecefc97011d847279a0795a6bed21e83` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.690113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecefc97011d847279a0795a6bed21e83)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.692154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.796952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:01.799827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:01.815025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2d654-50a5-4aaa-804b-f56edb66bdbf service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.829929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecefc97011d847279a0795a6bed21e83', 'request_id': 'req-d592adda-aa42-4fbf-9982-8648bed3f7e6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:01.842078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecefc97011d847279a0795a6bed21e83', 'request_id': 'req-d592adda-aa42-4fbf-9982-8648bed3f7e6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:01.855215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecefc97011d847279a0795a6bed21e83', 'request_id': 'req-d592adda-aa42-4fbf-9982-8648bed3f7e6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:01.878102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] The token cache is being invalidate because project ecefc97011d847279a0795a6bed21e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:01.878373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:01.879060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d592adda-aa42-4fbf-9982-8648bed3f7e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:01.884623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.885350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.885739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.886814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.887836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 989/1993] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:01 2022] DELETE /identity/v3/projects/ecefc97011d847279a0795a6bed21e83 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:01.897740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.898031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.898309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.899597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:01.915773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:01.940191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:01.942077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.079494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.092898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 990/1994] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:02.092898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.094048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.098520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.099981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a72a2f-6828-4adc-b0ce-5ab988ce4122 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.109297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4e05-7483-4ec8-aa2b-882c5fefba5f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.110556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1005/1995] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:02.111667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.111950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.112223 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.113918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.114652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.114998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.115323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] PATH_INFO: `/v3/users/8972d3c7ac5d4fcc8c4e937a5e12db90` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.116492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8972d3c7ac5d4fcc8c4e937a5e12db90)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.118583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.142921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:02.144189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.154402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1006/1996] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:02.159951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.172814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768382a7-2856-40c8-8769-05224f7a6e7e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.177042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.177660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.178020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.178270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.179294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:02.179602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.181407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.189492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8972d3c7ac5d4fcc8c4e937a5e12db90'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:02.199440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1007/1997] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:02.199609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8972d3c7ac5d4fcc8c4e937a5e12db90'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:02.200212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8972d3c7ac5d4fcc8c4e937a5e12db90'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:02.200650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.202590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.210035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8972d3c7ac5d4fcc8c4e937a5e12db90'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:02.217869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a137e3-1b73-4009-8e05-99d1d82f770b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.222784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.223391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.223669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.223943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.224567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.225375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-75e12733-8fdc-4e59-b2bb-d6f428c975dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8972d3c7ac5d4fcc8c4e937a5e12db90'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:02.226540 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.232663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 991/1998] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:02 2022] DELETE /identity/v3/users/8972d3c7ac5d4fcc8c4e937a5e12db90 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:02.237308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e4240-bfec-4640-ae90-440a6533358f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.237593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e4240-bfec-4640-ae90-440a6533358f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.237991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e4240-bfec-4640-ae90-440a6533358f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.262862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1008/1999] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:02.263058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-367e4240-bfec-4640-ae90-440a6533358f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:02.263772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.264181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-367e4240-bfec-4640-ae90-440a6533358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.343732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 992/2000] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:02.347939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92408be2-c291-4fce-a262-ca196747e12f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.354241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e4240-bfec-4640-ae90-440a6533358f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.356390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.356955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-367e4240-bfec-4640-ae90-440a6533358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.357425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.357800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.358045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.358694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.360660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.381472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.394981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1009/2001] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:02.399409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.414555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b7b39-b00d-4fa2-b56d-48d79215785a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.419992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.420604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.420936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.421303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.422080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.423706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.435606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e4240-bfec-4640-ae90-440a6533358f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.436715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1010/2002] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:02.442122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.443043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.443818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.444194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.444672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.457260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.459060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.463203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783f36-709c-41d1-be3b-faf0692aafa1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.473608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 993/2003] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:02.474091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e39c1f-c97d-4ffd-a79f-e7f6979e217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.474821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e39c1f-c97d-4ffd-a79f-e7f6979e217e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.475117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e39c1f-c97d-4ffd-a79f-e7f6979e217e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.475385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e39c1f-c97d-4ffd-a79f-e7f6979e217e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.476688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e39c1f-c97d-4ffd-a79f-e7f6979e217e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.478334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e39c1f-c97d-4ffd-a79f-e7f6979e217e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.502283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1011/2004] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:02.503625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.521485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460af34-d4d9-479c-bcff-7562aef273fb service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.534599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.535299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.535576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.535849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] PATH_INFO: `/v3/projects/57a9714e40d94a2ab2fada6ad1f6184a/users/e9426f8949d54a6ab6aa6cc6114147d5/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.561246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a9714e40d94a2ab2fada6ad1f6184a, user_id=e9426f8949d54a6ab6aa6cc6114147d5, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.565386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.589061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 994/2005] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:02 2022] PUT /identity/v3/projects/57a9714e40d94a2ab2fada6ad1f6184a/users/e9426f8949d54a6ab6aa6cc6114147d5/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:02.602946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.603967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.604433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.663464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] MFA Rules not processed for user `e9426f8949d54a6ab6aa6cc6114147d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:02.665232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.754657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1012/2006] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:02.760125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.873844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d555-baef-4e46-a04e-469816464b77 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.879281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.879956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.880318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.880722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.881880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.894907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.901376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.904373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.919406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.964975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22e138-01be-40a9-ba2f-dfb2dc8bd89f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.973050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.974102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.974588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.975082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] PATH_INFO: `/v3/projects/01f101a51c724982ab83002f89d59829` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.986214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 995/2007] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:02.987313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f101a51c724982ab83002f89d59829)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.990440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.999860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.095307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f101a51c724982ab83002f89d59829', 'request_id': 'req-dbefb635-ccef-426f-b7a4-25ae6786c06a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.110032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f101a51c724982ab83002f89d59829', 'request_id': 'req-dbefb635-ccef-426f-b7a4-25ae6786c06a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.124035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f101a51c724982ab83002f89d59829', 'request_id': 'req-dbefb635-ccef-426f-b7a4-25ae6786c06a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.150507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad89c-2bc0-4922-b382-bce38aafec7e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.153118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] The token cache is being invalidate because project 01f101a51c724982ab83002f89d59829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:03.153795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.154683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.157618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1013/2008] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:02 2022] DELETE /identity/v3/projects/01f101a51c724982ab83002f89d59829 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:03.161689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.163491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:03.218138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:03.319677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefb635-ccef-426f-b7a4-25ae6786c06a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.323663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.326514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.326984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.327954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.328604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.330585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.335409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.336243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.336663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.336970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] PATH_INFO: `/v3/users/497a903d6f34449c87f0284ed5c84454` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.338535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497a903d6f34449c87f0284ed5c84454)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.344049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.396324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 996/2009] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/users => generated 299 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:03.400453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.420542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:03.430130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de42cc-7ba8-4289-9637-bb2c7769d8d6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.452187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.454880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.455169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.455450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.456112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.459756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.473635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497a903d6f34449c87f0284ed5c84454'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.486428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 997/2010] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:03.494449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497a903d6f34449c87f0284ed5c84454'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.497823 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.522037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632302d-400b-47d4-acfb-473cf38afb7f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.525682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497a903d6f34449c87f0284ed5c84454'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.526827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497a903d6f34449c87f0284ed5c84454'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.533734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.534454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.534730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.535001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] PATH_INFO: `/v3/projects/c65c8317ba6b468e99c5f2df36f6f84b/users/8bc9b975ef8d4d6f906999d6f210812e/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.544134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497a903d6f34449c87f0284ed5c84454'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:03.556555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65c8317ba6b468e99c5f2df36f6f84b, user_id=8bc9b975ef8d4d6f906999d6f210812e, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.560661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.561798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1014/2011] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:03 2022] DELETE /identity/v3/users/497a903d6f34449c87f0284ed5c84454 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:03.586798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 998/2012] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:03 2022] PUT /identity/v3/projects/c65c8317ba6b468e99c5f2df36f6f84b/users/8bc9b975ef8d4d6f906999d6f210812e/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:03.591621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.594113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:03.603272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:03.709366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd8114-1443-4fa0-92cd-5f14cd2d2e60 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.784655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c91b6-db76-4fd2-ab93-8e3f5c928a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.785400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c91b6-db76-4fd2-ab93-8e3f5c928a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.785756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c91b6-db76-4fd2-ab93-8e3f5c928a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.786102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c91b6-db76-4fd2-ab93-8e3f5c928a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.786767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c91b6-db76-4fd2-ab93-8e3f5c928a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.788621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c91b6-db76-4fd2-ab93-8e3f5c928a99 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.804820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1015/2013] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:03.806708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.817924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915717c-3079-45bf-b5d9-2fe6d75b5a42 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.823152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.824687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.825165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.825959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] PATH_INFO: `/v3/projects/c65c8317ba6b468e99c5f2df36f6f84b/users/8bc9b975ef8d4d6f906999d6f210812e/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.827569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65c8317ba6b468e99c5f2df36f6f84b, user_id=8bc9b975ef8d4d6f906999d6f210812e, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.830499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.843695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 999/2014] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:03 2022] PUT /identity/v3/projects/c65c8317ba6b468e99c5f2df36f6f84b/users/8bc9b975ef8d4d6f906999d6f210812e/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:03.861568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.862364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.862647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.895202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] MFA Rules not processed for user `8bc9b975ef8d4d6f906999d6f210812e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:03.896868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.015325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1016/2015] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:04.057661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.057661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.059037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.062126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.116227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.151795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240e71-00a4-4bb7-90e2-cf650b4be920 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.155448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4c655-84b0-424f-b614-97101aad6c99 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.156931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.157562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.157831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.159878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.162046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.162236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.163220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.163694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.164146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] PATH_INFO: `/v3/projects/228edafff7b54a00a734dfd387294045` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.165670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228edafff7b54a00a734dfd387294045)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.168392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.242300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.242960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228edafff7b54a00a734dfd387294045', 'request_id': 'req-1385ec13-eba4-4df9-8f37-b500f7201550'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:04.243810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1e0b7-6848-46e1-9926-0eaf8d4ad180 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.251797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228edafff7b54a00a734dfd387294045', 'request_id': 'req-1385ec13-eba4-4df9-8f37-b500f7201550'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:04.254433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1017/2016] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:04.260987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228edafff7b54a00a734dfd387294045', 'request_id': 'req-1385ec13-eba4-4df9-8f37-b500f7201550'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:04.270352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.274531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] The token cache is being invalidate because project 228edafff7b54a00a734dfd387294045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:04.275183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:04.285580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1385ec13-eba4-4df9-8f37-b500f7201550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:04.287200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1000/2017] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:04 2022] DELETE /identity/v3/projects/228edafff7b54a00a734dfd387294045 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:04.293703 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.294062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.295250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.314872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:04.316101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.385671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1018/2018] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:04.394112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.394387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.394722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.419659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:04.422023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.443674 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1001/2019] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:04.448088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.464101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df7c84-7fd2-43b3-8c45-c0a3c6b4b8e2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.491786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.499100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.499378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.499656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.500386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:04.500686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.502928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.522182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1019/2020] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:04.526518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.538295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790a4fc-5305-4192-b193-571bd15ccada None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.549515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.550347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.550838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.552119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.552927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.555048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.599421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1002/2021] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:04 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:04.600291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.612424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a630176-c828-42f3-a530-3a2d8cc1554e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.726200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.727027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.727556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.727944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.728792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.733125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.787790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1020/2022] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:04 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:04.792469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.810230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa545ad8-7b62-4c7a-9fb2-17def455409f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.816839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.817345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.817725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.818100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.819089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.821035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.835469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1003/2023] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:04.840503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.863439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.871641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380dd52-75bf-4302-92a2-ac8b3797b79a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.884020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790bc72-4fce-42c0-a617-a11fa3111a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.885058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790bc72-4fce-42c0-a617-a11fa3111a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.885631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790bc72-4fce-42c0-a617-a11fa3111a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.887555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790bc72-4fce-42c0-a617-a11fa3111a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.889148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790bc72-4fce-42c0-a617-a11fa3111a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.894809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790bc72-4fce-42c0-a617-a11fa3111a18 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.929586 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1021/2024] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:04.931052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.946764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b776df-cb3f-414f-a54c-6ba6b6b1e59e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.955914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.956983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.957506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.957862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] PATH_INFO: `/v3/projects/6e13b1d9da8d45afb00b1eb6750c42e1/users/f33c27f871094be99cf93b4f344f71b1/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.006220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e13b1d9da8d45afb00b1eb6750c42e1, user_id=f33c27f871094be99cf93b4f344f71b1, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.013408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.044258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1004/2025] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:04 2022] PUT /identity/v3/projects/6e13b1d9da8d45afb00b1eb6750c42e1/users/f33c27f871094be99cf93b4f344f71b1/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:05.055567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.056013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.056163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.099773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] MFA Rules not processed for user `f33c27f871094be99cf93b4f344f71b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:05.101366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.255031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1022/2026] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:05.373864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.375832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.450017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d185050-356b-495b-a844-3d17c6f0acbc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.453617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.454218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.454493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.454760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.463243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.465946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.474581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1005/2027] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:05.478273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.479757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.494750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.574571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677652fe-797d-447a-9ba7-c141dfae5b0f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.585083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.585083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.585301 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.585677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] PATH_INFO: `/v3/users/b94b9f8a83374c108bc80083213aea2f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.588164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94b9f8a83374c108bc80083213aea2f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.590931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.602943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7f3bb-381a-45d9-a764-21f6bff66e15 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.610297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.611036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.611321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.611596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] PATH_INFO: `/v3/users/f33c27f871094be99cf93b4f344f71b1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.612797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33c27f871094be99cf93b4f344f71b1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.614742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.658102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94b9f8a83374c108bc80083213aea2f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.666339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.667275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94b9f8a83374c108bc80083213aea2f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.676308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33c27f871094be99cf93b4f344f71b1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.685711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33c27f871094be99cf93b4f344f71b1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.686069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94b9f8a83374c108bc80083213aea2f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.686937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94b9f8a83374c108bc80083213aea2f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.687186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33c27f871094be99cf93b4f344f71b1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.702412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33c27f871094be99cf93b4f344f71b1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.709685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94b9f8a83374c108bc80083213aea2f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.720400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2c41d295-db12-4428-8b51-6e6be1eb3ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33c27f871094be99cf93b4f344f71b1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.726153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1023/2028] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:05 2022] DELETE /identity/v3/users/b94b9f8a83374c108bc80083213aea2f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:05.727087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.728726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1006/2029] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:05 2022] DELETE /identity/v3/users/f33c27f871094be99cf93b4f344f71b1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:05.732870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f3ebf-82c1-49c2-9924-9ae5765ab32e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.732870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f3ebf-82c1-49c2-9924-9ae5765ab32e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.732870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f3ebf-82c1-49c2-9924-9ae5765ab32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.733695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.739613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.756853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5f1f3ebf-82c1-49c2-9924-9ae5765ab32e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:05.758106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5f1f3ebf-82c1-49c2-9924-9ae5765ab32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.815505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9dd49-02bb-4bb3-b76b-0aa9fc233178 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.832751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1007/2030] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:05.834542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.834838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.835113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.860865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:05.863555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.871867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1008/2031] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:05.872699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.874348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:05.911259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.912369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.912745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.913026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] PATH_INFO: `/v3/users/a2ce922a4b7e483ea253a03a0072eb9d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.914405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ce922a4b7e483ea253a03a0072eb9d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.916565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.940840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e45c-b8be-47b3-a2fd-4f12e8f5eefe None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.944350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.945159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.945500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.946380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.958123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.959923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.972073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ce922a4b7e483ea253a03a0072eb9d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.972850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1009/2032] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:05.974098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.980899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ce922a4b7e483ea253a03a0072eb9d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.985514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b439d4-7c96-40f2-8d15-581e0382005d service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.990183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.990971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.991259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.991592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.995888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ce922a4b7e483ea253a03a0072eb9d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:05.996611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ce922a4b7e483ea253a03a0072eb9d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.005132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.005498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be371698-0a4c-4963-89cd-008807863b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ce922a4b7e483ea253a03a0072eb9d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.007418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.012674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1024/2033] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:05 2022] DELETE /identity/v3/users/a2ce922a4b7e483ea253a03a0072eb9d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:06.016951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.017241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.018034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.024961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1010/2034] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:06.039309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:06.040559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.066851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.122984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1025/2035] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:06.141728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.150648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95ca4a-4617-42b2-aee4-a6dcef882153 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.215956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.233502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.234459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.234736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.235099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.265265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.266829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.301325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1011/2036] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:06.736415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.737919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.750183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09788dc3-5545-4aa9-9e98-fe572644d289 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.772813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.773585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.773959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.774327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] PATH_INFO: `/v3/projects/d2c1caad11964d5d9ac9222707190c78` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.775619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c1caad11964d5d9ac9222707190c78)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.780328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.877826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.879345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.895630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35cc5f-2dcb-4acf-8180-28be9b189106 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.898925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c1caad11964d5d9ac9222707190c78', 'request_id': 'req-1321501a-06ee-4bed-ac4f-54c791995ed4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.910060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c1caad11964d5d9ac9222707190c78', 'request_id': 'req-1321501a-06ee-4bed-ac4f-54c791995ed4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.919410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c1caad11964d5d9ac9222707190c78', 'request_id': 'req-1321501a-06ee-4bed-ac4f-54c791995ed4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.937794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] The token cache is being invalidate because project d2c1caad11964d5d9ac9222707190c78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:06.944986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.944986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:06.944986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1026/2037] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:06 2022] DELETE /identity/v3/projects/d2c1caad11964d5d9ac9222707190c78 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:06.944986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.955189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.991497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.992288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.992665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.993064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] PATH_INFO: `/v3/projects/6e13b1d9da8d45afb00b1eb6750c42e1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.994345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e13b1d9da8d45afb00b1eb6750c42e1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.997152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.056344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e13b1d9da8d45afb00b1eb6750c42e1', 'request_id': 'req-48aa19d7-3ca2-4668-996d-5fcad66b03f3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.068869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e13b1d9da8d45afb00b1eb6750c42e1', 'request_id': 'req-48aa19d7-3ca2-4668-996d-5fcad66b03f3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.074310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321501a-06ee-4bed-ac4f-54c791995ed4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.079454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e13b1d9da8d45afb00b1eb6750c42e1', 'request_id': 'req-48aa19d7-3ca2-4668-996d-5fcad66b03f3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.100088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] The token cache is being invalidate because project 6e13b1d9da8d45afb00b1eb6750c42e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:07.100682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.103063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-48aa19d7-3ca2-4668-996d-5fcad66b03f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.106460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1012/2038] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:06 2022] DELETE /identity/v3/projects/6e13b1d9da8d45afb00b1eb6750c42e1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:07.111316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6280e-0fad-4811-ae80-cf15f218343f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.111715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6280e-0fad-4811-ae80-cf15f218343f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.112085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6280e-0fad-4811-ae80-cf15f218343f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.139289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-92e6280e-0fad-4811-ae80-cf15f218343f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:07.140849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-92e6280e-0fad-4811-ae80-cf15f218343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.167603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.169607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.169890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.170189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] PATH_INFO: `/v3/users/e25776edcca04c648ee03d35f10c10c9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.171528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25776edcca04c648ee03d35f10c10c9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.175269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.187011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.240935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1013/2039] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:07.247113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.247401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.247699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.252042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25776edcca04c648ee03d35f10c10c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.263209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25776edcca04c648ee03d35f10c10c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.274522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:07.275735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.281744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25776edcca04c648ee03d35f10c10c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.282454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25776edcca04c648ee03d35f10c10c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.290064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1014/2040] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:07.296873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.301428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25776edcca04c648ee03d35f10c10c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.304221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c0206-5d18-477e-aed9-9246afaf5851 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.311515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.312143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.312426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.312810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1027/2041] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:06 2022] DELETE /identity/v3/users/e25776edcca04c648ee03d35f10c10c9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:07.314249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.314930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.315245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.322893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.328944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.360797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1015/2042] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.365427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.367729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.378444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.380547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.420610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f0bb-4b00-43c5-8da9-2c859cb15160 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.507025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d351-25ed-4d32-bec3-25d04d48a7ed admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.523576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.524583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.525110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.525529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.536570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.537300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.537583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.537860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] PATH_INFO: `/v3/projects/4d10a6d4c59c4bd4b5f49fe0fec3772c` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.539086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d10a6d4c59c4bd4b5f49fe0fec3772c)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.541042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.541568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.546525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.592621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1028/2043] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:07 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:07.593805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.634424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d10a6d4c59c4bd4b5f49fe0fec3772c', 'request_id': 'req-49e814c4-81bc-4d84-aeac-51d2a093a367'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.646169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d10a6d4c59c4bd4b5f49fe0fec3772c', 'request_id': 'req-49e814c4-81bc-4d84-aeac-51d2a093a367'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.665597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d10a6d4c59c4bd4b5f49fe0fec3772c', 'request_id': 'req-49e814c4-81bc-4d84-aeac-51d2a093a367'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.702889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] The token cache is being invalidate because project 4d10a6d4c59c4bd4b5f49fe0fec3772c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:07.703466 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.704262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.706343 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1016/2044] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:07 2022] DELETE /identity/v3/projects/4d10a6d4c59c4bd4b5f49fe0fec3772c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:07.707632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87419589-50a1-4091-b60b-cd98d5d682a9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.710161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.712026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.724056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.831760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.833485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.833725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.836795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.836795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.837918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.895671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1029/2045] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:07 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:07.896887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.899634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.919401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c696f6-8049-45f7-beed-7338bbfa7052 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.925612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.926306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.926919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.927214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] PATH_INFO: `/v3/projects/0bca7b0115544e1d86486b7b760c4524` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.928450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bca7b0115544e1d86486b7b760c4524)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.930873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.995629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bca7b0115544e1d86486b7b760c4524', 'request_id': 'req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:07.996391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e814c4-81bc-4d84-aeac-51d2a093a367 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.008925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bca7b0115544e1d86486b7b760c4524', 'request_id': 'req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.021375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bca7b0115544e1d86486b7b760c4524', 'request_id': 'req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.039038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] The token cache is being invalidate because project 0bca7b0115544e1d86486b7b760c4524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:08.039501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.042663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.045618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1030/2046] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:07 2022] DELETE /identity/v3/projects/0bca7b0115544e1d86486b7b760c4524 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:08.046368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.047995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:08.085910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.143995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.144651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.144961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.145236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] PATH_INFO: `/v3/users/f0d8e50508be4e45b53e809b31762647` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.153305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d8e50508be4e45b53e809b31762647)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.155487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.195174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f78e-f6e3-4690-a1ed-6b0ef615cb4e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.204476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.205484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.205964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.206265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.206834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.211092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.240175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1031/2047] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:08.244982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d8e50508be4e45b53e809b31762647'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.247734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.254560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:08.262409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d8e50508be4e45b53e809b31762647'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.263852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d8e50508be4e45b53e809b31762647'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.286248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d8e50508be4e45b53e809b31762647'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.320225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94caabd-141f-4a41-958c-f6a25fef3d17 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.325226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d8e50508be4e45b53e809b31762647'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.336829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.337795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.338228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.338684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] PATH_INFO: `/v3/users/a5ecb36f97c94866a25ac85f807736eb` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.339171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1017/2048] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:07 2022] DELETE /identity/v3/users/f0d8e50508be4e45b53e809b31762647 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:08.343048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ecb36f97c94866a25ac85f807736eb)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.345571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.351171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.358493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:08.377289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.459916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ecb36f97c94866a25ac85f807736eb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.479131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ada276-08b5-48f1-8b47-efbb49618048 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.480209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ecb36f97c94866a25ac85f807736eb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.500930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8770ed-8c5a-4e57-a157-af69bda09174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.501822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8770ed-8c5a-4e57-a157-af69bda09174 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.502214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8770ed-8c5a-4e57-a157-af69bda09174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.502496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8770ed-8c5a-4e57-a157-af69bda09174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.503457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8770ed-8c5a-4e57-a157-af69bda09174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.504471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ecb36f97c94866a25ac85f807736eb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.505890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ecb36f97c94866a25ac85f807736eb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.508001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8770ed-8c5a-4e57-a157-af69bda09174 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.518876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ecb36f97c94866a25ac85f807736eb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:08.531582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1032/2049] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:08 2022] DELETE /identity/v3/users/a5ecb36f97c94866a25ac85f807736eb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:08.536321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1018/2050] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:08.540610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.542209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:08.665273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc679b1d-2bf7-4931-ace2-5cb2fd484731 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.672042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.672997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.673428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.673853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] PATH_INFO: `/v3/projects/cfe796b4da6748b6b44eaf9cd42d98ac/users/53730ff461b142208d249a93842934f1/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.694828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe796b4da6748b6b44eaf9cd42d98ac, user_id=53730ff461b142208d249a93842934f1, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.698270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.723183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1033/2051] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:08 2022] PUT /identity/v3/projects/cfe796b4da6748b6b44eaf9cd42d98ac/users/53730ff461b142208d249a93842934f1/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:08.734181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.734496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.734778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.809518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] MFA Rules not processed for user `53730ff461b142208d249a93842934f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:08.810765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:08.910112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.990602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1019/2052] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:08.991982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.136487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.142503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.163864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ea94e-3969-4f53-8f5e-d7b339163cf1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.172356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a3840-9781-40be-bc03-9908faed19d2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.180505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.181788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.182180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.182550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.183206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.187416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.187867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.188290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.188611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.188878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] PATH_INFO: `/v3/projects/2f0de9c4dcc0434aa496d79275fe72af` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.191170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0de9c4dcc0434aa496d79275fe72af)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.197326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.237940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.249022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1034/2053] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:08 2022] POST /identity/v3/projects => generated 375 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:09.254311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.258922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.284491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0de9c4dcc0434aa496d79275fe72af', 'request_id': 'req-165ee6f4-9936-4871-be86-f220b907b1d5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.295960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0de9c4dcc0434aa496d79275fe72af', 'request_id': 'req-165ee6f4-9936-4871-be86-f220b907b1d5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.309293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0de9c4dcc0434aa496d79275fe72af', 'request_id': 'req-165ee6f4-9936-4871-be86-f220b907b1d5'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.346283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] The token cache is being invalidate because project 2f0de9c4dcc0434aa496d79275fe72af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:09.346697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.354935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.359462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1020/2054] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:09 2022] DELETE /identity/v3/projects/2f0de9c4dcc0434aa496d79275fe72af => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:09.363536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.366276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.383516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669355aa-0b5d-4a6e-b9fd-4d76a1ecc723 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.492595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.493480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.493991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.494373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] PATH_INFO: `/v3/projects/2ef5a27b0cdf458a81323ec094f47d21` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.495734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef5a27b0cdf458a81323ec094f47d21)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.499489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.519314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee6f4-9936-4871-be86-f220b907b1d5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.526773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af102733-e5c8-4ed8-9c6d-d91b6fd3208c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.528159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af102733-e5c8-4ed8-9c6d-d91b6fd3208c admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.529044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af102733-e5c8-4ed8-9c6d-d91b6fd3208c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.529428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af102733-e5c8-4ed8-9c6d-d91b6fd3208c admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.530366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af102733-e5c8-4ed8-9c6d-d91b6fd3208c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.534182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af102733-e5c8-4ed8-9c6d-d91b6fd3208c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.592354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1021/2055] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:09.594514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.595368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.595906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.609205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef5a27b0cdf458a81323ec094f47d21', 'request_id': 'req-fb906c51-7121-4862-986b-7e882420459b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.618048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef5a27b0cdf458a81323ec094f47d21', 'request_id': 'req-fb906c51-7121-4862-986b-7e882420459b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.628758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef5a27b0cdf458a81323ec094f47d21', 'request_id': 'req-fb906c51-7121-4862-986b-7e882420459b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.643177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:09.643744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] The token cache is being invalidate because project 2ef5a27b0cdf458a81323ec094f47d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:09.644045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.644494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.645189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:09.647351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1035/2056] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:09 2022] DELETE /identity/v3/projects/2ef5a27b0cdf458a81323ec094f47d21 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:09.648385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.653218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.702987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.721128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.728860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb906c51-7121-4862-986b-7e882420459b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.735427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.736050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.736331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.736604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.738459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.740797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1022/2057] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:09.741666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.743001 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.756655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6c9f2-8195-40bf-b9c6-4fdf8c7f511b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.764189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07227eee-eee7-4027-9234-3afbb141045e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.764978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07227eee-eee7-4027-9234-3afbb141045e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.765360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07227eee-eee7-4027-9234-3afbb141045e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.765741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07227eee-eee7-4027-9234-3afbb141045e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.774657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07227eee-eee7-4027-9234-3afbb141045e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.774657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07227eee-eee7-4027-9234-3afbb141045e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.781516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1023/2058] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:09.786548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0cd25-84bd-42ae-a273-63c317258870 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.787089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0cd25-84bd-42ae-a273-63c317258870 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.787463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0cd25-84bd-42ae-a273-63c317258870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.812281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ceb0cd25-84bd-42ae-a273-63c317258870 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:09.812281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ceb0cd25-84bd-42ae-a273-63c317258870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.821403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1024/2059] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:09.822890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0966-0fa7-4cfa-8fc1-4048667bbe70 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.823281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0966-0fa7-4cfa-8fc1-4048667bbe70 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.823662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0966-0fa7-4cfa-8fc1-4048667bbe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.837426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.838888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.846035 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c23a0966-0fa7-4cfa-8fc1-4048667bbe70 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:09.847620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c23a0966-0fa7-4cfa-8fc1-4048667bbe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.850022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1036/2060] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:09.850990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.857897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1025/2061] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:09.859471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.859829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.860202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.876145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05718b-ba43-4827-bb50-fe515b3aa987 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.882156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.882412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:09.882898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.883269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.883629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.883917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.884406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.886239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.893071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1026/2062] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:09.893998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.909193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37040428-885e-458d-a2fb-2ce326c503d0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.914728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.915348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.919976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1037/2063] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:09.920173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.920173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.920316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.920387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:09.920387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.920387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.934858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1027/2064] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:09.935678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.953018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e7ff8-bec5-403f-b124-281a303a1276 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.958829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.959431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.959714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.959989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.960712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:09.960875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deaf5d-62d1-4e7a-a0c5-2843fe7ec3fb admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.961062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.962664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.973336 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.974007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.974295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.974564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] PATH_INFO: `/v3/projects/7bed4dc04a29497a89c587ffe2ba34ee/users/7d579cd2393c48ceafb50dd2e3508298/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.979040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1038/2065] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:09.980055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.992334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8eef5-1c0e-49fa-b9b6-6637dab8009a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.000471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.001568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.002035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.002472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.003269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.007092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.018729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bed4dc04a29497a89c587ffe2ba34ee, user_id=7d579cd2393c48ceafb50dd2e3508298, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.026755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.041242 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1039/2066] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:10.042493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.047934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1028/2067] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:09 2022] PUT /identity/v3/projects/7bed4dc04a29497a89c587ffe2ba34ee/users/7d579cd2393c48ceafb50dd2e3508298/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:10.049843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.148318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e524f-9552-4db0-9bb6-861c09b2b973 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.177374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c104-7063-4b56-91c7-e069735f3b6f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.185809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e1069-1cb3-4953-8e75-92e8bb440f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.186637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e1069-1cb3-4953-8e75-92e8bb440f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.193106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e1069-1cb3-4953-8e75-92e8bb440f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.193557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e1069-1cb3-4953-8e75-92e8bb440f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.194399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e1069-1cb3-4953-8e75-92e8bb440f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.201597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e1069-1cb3-4953-8e75-92e8bb440f3e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.261322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.263615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.264140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.264574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.265421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.271230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.283609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1040/2068] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:10.285944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.286900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.287528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.317023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1029/2069] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:10.317023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.322072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa36d-a250-486c-856a-d41b8df1d1c5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.345078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.362095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] MFA Rules not processed for user `7d579cd2393c48ceafb50dd2e3508298`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:10.364229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:10.409481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.449921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.450671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.450944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.451214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.451844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.453832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.486922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1030/2070] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:10.486922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.501169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8096da5-714c-4a1d-95cd-7213b06e8f11 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.508181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.508978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.513437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.513437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.513437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.513437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.514497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1041/2071] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:10.515893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.528918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf49d4-c77e-4470-a3fd-ba6ebec97211 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.533939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.534977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.535396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.535817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.536606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.538692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.555069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1042/2072] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:10.555069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.555303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1031/2073] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:10.556370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.568982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af199-ab53-40a5-a231-b83de144226b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.570488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba162e-77d5-4d7c-b401-94830e1d6e62 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.574915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.575540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.575652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.576005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.576568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.576689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.576808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.577071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.577440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.577855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.579506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.581275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.592463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1032/2074] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:10.593593 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.613533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5db321-8f78-48a2-a5a8-db53e460785d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.620841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1043/2075] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:10.621004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.621004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.621306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.621962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.621962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.621962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.624229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.640155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1033/2076] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:10.641076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.651245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a756f-24cb-44ae-83e1-c10f1042ab8e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.708303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061a24-a99b-4844-b6e2-56c3f18903f8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.713564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.714471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.715401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.715984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] PATH_INFO: `/v3/projects/acb6cb6c88774f6bbee061efd7618719/users/f6e3ba7d542a4e19b65cf569b6dd398f/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.721463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.721874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.722153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.722431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.723066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.725023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.735240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb6cb6c88774f6bbee061efd7618719, user_id=f6e3ba7d542a4e19b65cf569b6dd398f, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.737750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.753715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1034/2077] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:10.753715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.759327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1044/2078] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:10 2022] PUT /identity/v3/projects/acb6cb6c88774f6bbee061efd7618719/users/f6e3ba7d542a4e19b65cf569b6dd398f/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:10.760048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.773186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0aa7a-fa52-4f7e-936e-0d2a55b6bacc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.836016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65478c4f-78a2-4c18-84e3-034df9e949fa admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.836533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.837287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.837621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.837975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] PATH_INFO: `/v3/projects/bedae026c26d40a18eb94996699d677f/users/9889da0d0c834440bb857d347ad87fcd/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.841666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238dc0b-c8eb-4036-b1c4-916f16e07253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.842283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238dc0b-c8eb-4036-b1c4-916f16e07253 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.842562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238dc0b-c8eb-4036-b1c4-916f16e07253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.842827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238dc0b-c8eb-4036-b1c4-916f16e07253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.843507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238dc0b-c8eb-4036-b1c4-916f16e07253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.845174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238dc0b-c8eb-4036-b1c4-916f16e07253 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.850804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.855234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedae026c26d40a18eb94996699d677f, user_id=9889da0d0c834440bb857d347ad87fcd, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.856650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1045/2079] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:10.858054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947274c-0945-474d-af70-6e84ed25875d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.858398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947274c-0945-474d-af70-6e84ed25875d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.858744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947274c-0945-474d-af70-6e84ed25875d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.858885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.866922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.875345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1035/2080] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:10 2022] PUT /identity/v3/projects/bedae026c26d40a18eb94996699d677f/users/9889da0d0c834440bb857d347ad87fcd/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:10.876152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.895414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3947274c-0945-474d-af70-6e84ed25875d None None] MFA Rules not processed for user `f6e3ba7d542a4e19b65cf569b6dd398f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:10.896699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3947274c-0945-474d-af70-6e84ed25875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:10.946932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd75e6-713c-4f9a-a913-3f3f19def671 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.951850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.952500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.952805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.953083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] PATH_INFO: `/v3/projects/8ba11eae063e421891d7a2c844750709/users/64c21cbee3c343a2a0d613d20d3ff1c9/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.963041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1046/2081] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:10.964332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.964624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.965524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.969947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba11eae063e421891d7a2c844750709, user_id=64c21cbee3c343a2a0d613d20d3ff1c9, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.972712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.989103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1036/2082] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:10 2022] PUT /identity/v3/projects/8ba11eae063e421891d7a2c844750709/users/64c21cbee3c343a2a0d613d20d3ff1c9/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:10.989911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.001627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] MFA Rules not processed for user `9889da0d0c834440bb857d347ad87fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:11.003144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:11.067992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da7315-d41d-49ed-a07e-9b4affbd2a01 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.071781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1047/2083] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:11.075948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.076818 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.077473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.077772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.078020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.078648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.080572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.083031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9770a-f2c1-4841-aca0-f1153d1cc70f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.088049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.088641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.088959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.089235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.089874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.091714 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.102571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1048/2084] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:11.103350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.109065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1037/2085] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:11.109750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.113879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028b082-340e-44ee-b27f-8346d6a21bc8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.171749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.174413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0add3e-3cfc-47d0-92eb-7d7b23a49d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.175035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0add3e-3cfc-47d0-92eb-7d7b23a49d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.175306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0add3e-3cfc-47d0-92eb-7d7b23a49d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.175575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0add3e-3cfc-47d0-92eb-7d7b23a49d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.175786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06d5a0-7c23-4c25-94fc-58e024c61c30 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.176203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0add3e-3cfc-47d0-92eb-7d7b23a49d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.178351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0add3e-3cfc-47d0-92eb-7d7b23a49d22 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.181974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.182665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.183013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.183691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] PATH_INFO: `/v3/projects/8ba11eae063e421891d7a2c844750709/users/64c21cbee3c343a2a0d613d20d3ff1c9/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.186495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba11eae063e421891d7a2c844750709, user_id=64c21cbee3c343a2a0d613d20d3ff1c9, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.189542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.197560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.201409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1038/2086] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:11 2022] PUT /identity/v3/projects/8ba11eae063e421891d7a2c844750709/users/64c21cbee3c343a2a0d613d20d3ff1c9/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:11.202747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.214699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1049/2087] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:11.214699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.214699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.214699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.244507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] MFA Rules not processed for user `64c21cbee3c343a2a0d613d20d3ff1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:11.245767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:11.269469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6b0a1-62ca-492b-8e91-da1acf74ee16 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.321400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1050/2088] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:11.322146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.335026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.335804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.336236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.336598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.337260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.339422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.362691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1039/2089] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:11.363584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.402175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280d5c0-7832-4a2b-839d-a1395bcb949d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.408936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.409537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.409813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.410084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.410706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.412678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.422082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d0ccd-a74c-4f6a-8b43-2506bfbe5818 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.426322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.426972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.428152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.428152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.429614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:11.435631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1051/2090] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:11.436601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.447488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae09a6d-e5d4-479f-96cf-fb8dd77ae71f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.452470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.453104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.453382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.453652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.454296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.456099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.467448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1052/2091] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:11.468404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.482637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86ec67-29de-482c-85fe-332b0cbcdcca admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.488470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.489293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.490103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.491076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.491392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.492847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.495563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.509527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.510866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1053/2092] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:11.511238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.511682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.521383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1040/2093] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:11.522102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.524063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3f86-14a7-480b-9f69-54fbd0596c2a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.533381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84957835-aa1a-4718-b453-62bc3036fa92 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.534487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.535153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.535430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.535705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] PATH_INFO: `/v3/projects/e2739c58b1274d5aa2fc0b76e55cb905/users/d2c4f2669d584c7eaaa8d27effd5b3c1/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.539531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.540214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.540490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.540803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] PATH_INFO: `/v3/projects/285476b69c6c4ed1b1349b9795127f7a/users/253d270b222149c29ddc58b8b7aad52d/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.555644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2739c58b1274d5aa2fc0b76e55cb905, user_id=d2c4f2669d584c7eaaa8d27effd5b3c1, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.558194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=285476b69c6c4ed1b1349b9795127f7a, user_id=253d270b222149c29ddc58b8b7aad52d, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.559222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.561529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.578277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1041/2094] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:11 2022] PUT /identity/v3/projects/285476b69c6c4ed1b1349b9795127f7a/users/253d270b222149c29ddc58b8b7aad52d/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:11.579314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1054/2095] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:11 2022] PUT /identity/v3/projects/e2739c58b1274d5aa2fc0b76e55cb905/users/d2c4f2669d584c7eaaa8d27effd5b3c1/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:11.598719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.598819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.633653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.695089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190db0c-d8f2-4ea2-84e1-37fb2caa1305 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.701209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.702153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.702491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.702678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.703588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.704085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debd3dc-f627-48b2-9be0-826d2ec71dac admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.705795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.711325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.711919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.712342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.713206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.713934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.715767 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.725408 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1055/2096] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:11.730029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.731472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1042/2097] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:11.732461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.744098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb73e8-cf9b-4c98-97f8-106a7f49326c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.744430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076737a2-3409-4eb9-ad9d-2c8dc004ba8a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.751172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3ba52-e2e5-4836-8419-1548b2b03a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.752150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3ba52-e2e5-4836-8419-1548b2b03a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.752482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3ba52-e2e5-4836-8419-1548b2b03a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.752652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb319e4b-866c-49d9-8f78-c268b9efb8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.752923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3ba52-e2e5-4836-8419-1548b2b03a2e admin admin] PATH_INFO: `/v3/projects/285476b69c6c4ed1b1349b9795127f7a/users/253d270b222149c29ddc58b8b7aad52d/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.753529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319e4b-866c-49d9-8f78-c268b9efb8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.753871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319e4b-866c-49d9-8f78-c268b9efb8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.754174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319e4b-866c-49d9-8f78-c268b9efb8dc admin admin] PATH_INFO: `/v3/projects/e2739c58b1274d5aa2fc0b76e55cb905/users/d2c4f2669d584c7eaaa8d27effd5b3c1/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.755194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3ba52-e2e5-4836-8419-1548b2b03a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=285476b69c6c4ed1b1349b9795127f7a, user_id=253d270b222149c29ddc58b8b7aad52d, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.756647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb319e4b-866c-49d9-8f78-c268b9efb8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2739c58b1274d5aa2fc0b76e55cb905, user_id=d2c4f2669d584c7eaaa8d27effd5b3c1, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.758086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3ba52-e2e5-4836-8419-1548b2b03a2e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.759702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb319e4b-866c-49d9-8f78-c268b9efb8dc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.773262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1056/2098] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:11 2022] PUT /identity/v3/projects/e2739c58b1274d5aa2fc0b76e55cb905/users/d2c4f2669d584c7eaaa8d27effd5b3c1/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:11.777171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1043/2099] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:11 2022] PUT /identity/v3/projects/285476b69c6c4ed1b1349b9795127f7a/users/253d270b222149c29ddc58b8b7aad52d/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:11.788112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.788316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.788611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.792668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.793610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.793839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.833791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] MFA Rules not processed for user `d2c4f2669d584c7eaaa8d27effd5b3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:11.835705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:11.837933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] MFA Rules not processed for user `253d270b222149c29ddc58b8b7aad52d`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:11.839308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:11.976748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1057/2100] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:12.011847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.015304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:12.039287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1044/2101] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:12.040070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.076028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.091480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b985c-e25a-440f-9ac9-64e9c36debc7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.095934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.097005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.097456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.097895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.109354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.110854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.116434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aebbb1-b8eb-4ab6-bc27-8239c8681e81 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.120347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.120665 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1058/2102] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:12.120985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.121364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.122251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.131991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.133499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.152448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1045/2103] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:12.261628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.271829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c1379-0bf6-4a2e-b8fa-0b6696a14425 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.275671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433b808-746e-417c-ba33-b3ee29aff2c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.276362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433b808-746e-417c-ba33-b3ee29aff2c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.276641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433b808-746e-417c-ba33-b3ee29aff2c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.276945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433b808-746e-417c-ba33-b3ee29aff2c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.313786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.343926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433b808-746e-417c-ba33-b3ee29aff2c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.345364 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433b808-746e-417c-ba33-b3ee29aff2c4 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.357402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1059/2104] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:12.731531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.795650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1801233-ba77-457b-9ef5-0329bded5bc1 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.800567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.801220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.801505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.801776 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] PATH_INFO: `/v3/users/9889da0d0c834440bb857d347ad87fcd` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.803016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9889da0d0c834440bb857d347ad87fcd)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.805031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.851663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9889da0d0c834440bb857d347ad87fcd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:12.859477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9889da0d0c834440bb857d347ad87fcd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:12.860168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9889da0d0c834440bb857d347ad87fcd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:12.866521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9889da0d0c834440bb857d347ad87fcd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:12.880174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9889da0d0c834440bb857d347ad87fcd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:12.886033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1046/2105] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:12 2022] DELETE /identity/v3/users/9889da0d0c834440bb857d347ad87fcd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:12.890840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.891281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.891621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.920328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:12.922034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:12.997635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1060/2106] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:13.008440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.009927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:13.093068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.098595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d5f5e-d348-461d-81c0-195b90aa6dac admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.105228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.106814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.107097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.107364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.117996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.119621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.140928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1047/2107] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:13.279676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.282721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:13.353744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a374ce-a6ae-4d60-9541-f5f84ffb3a9b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.357977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.358742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.359393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.359393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.361532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:13.454293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.456029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.467130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.468015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1061/2108] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:13.472904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:13.500082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3934c-6b89-444f-911a-f7368b092d81 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.506218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.506916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.507298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.507640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] PATH_INFO: `/v3/projects/bedae026c26d40a18eb94996699d677f` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.508870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bedae026c26d40a18eb94996699d677f)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.510914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.563383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bedae026c26d40a18eb94996699d677f', 'request_id': 'req-322585e1-c1a8-4cd7-baec-79cb16379928'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.570855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bedae026c26d40a18eb94996699d677f', 'request_id': 'req-322585e1-c1a8-4cd7-baec-79cb16379928'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.582129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bedae026c26d40a18eb94996699d677f', 'request_id': 'req-322585e1-c1a8-4cd7-baec-79cb16379928'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.598093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] The token cache is being invalidate because project bedae026c26d40a18eb94996699d677f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:13.598427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.599144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.600698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1048/2109] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:13 2022] DELETE /identity/v3/projects/bedae026c26d40a18eb94996699d677f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:13.604802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.617399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b249-f600-4741-b8d1-7c16da060c4e service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.645821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.685461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.686276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.686854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.687106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] PATH_INFO: `/v3/users/253d270b222149c29ddc58b8b7aad52d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.688276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253d270b222149c29ddc58b8b7aad52d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.690515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.759462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253d270b222149c29ddc58b8b7aad52d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.769788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253d270b222149c29ddc58b8b7aad52d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.789403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253d270b222149c29ddc58b8b7aad52d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.793994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253d270b222149c29ddc58b8b7aad52d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.803735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253d270b222149c29ddc58b8b7aad52d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:13.810336 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1062/2110] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:13 2022] DELETE /identity/v3/users/253d270b222149c29ddc58b8b7aad52d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:14.141995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.145518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.189811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322585e1-c1a8-4cd7-baec-79cb16379928 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.288510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.289190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.289487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.289762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] PATH_INFO: `/v3/projects/285476b69c6c4ed1b1349b9795127f7a` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.291743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=285476b69c6c4ed1b1349b9795127f7a)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.296311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.326004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.328047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.376257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285476b69c6c4ed1b1349b9795127f7a', 'request_id': 'req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:14.384340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285476b69c6c4ed1b1349b9795127f7a', 'request_id': 'req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:14.396669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285476b69c6c4ed1b1349b9795127f7a', 'request_id': 'req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:14.398010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b17e94-e639-4b10-b3c1-9fb005ee6d35 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.431039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] The token cache is being invalidate because project 285476b69c6c4ed1b1349b9795127f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:14.431264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:14.431957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ad7a4f9d-cc98-447a-a47e-70b4fb76b2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:14.433690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1049/2111] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:14 2022] DELETE /identity/v3/projects/285476b69c6c4ed1b1349b9795127f7a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:14.440165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57185e3-6950-462d-957d-216c754ab9e1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.440831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57185e3-6950-462d-957d-216c754ab9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.441238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57185e3-6950-462d-957d-216c754ab9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.450254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.473375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c57185e3-6950-462d-957d-216c754ab9e1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:14.474817 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c57185e3-6950-462d-957d-216c754ab9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.476252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.477133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.477414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.477691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.479698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.510122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.594289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.596316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.599415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1050/2112] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:14.605130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.607363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.607640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.615724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1063/2113] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:14.632742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:14.633968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.644436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1051/2114] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:14.649198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.664314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ce96a-e8f9-46b1-948e-1dfe1cb316c4 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.671642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.672718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.672718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.672718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.674791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:14.674791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.675645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.690145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1064/2115] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:14.694734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.707537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8d58-fea6-4fb8-ab12-1070b153f1cb None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.712283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.712934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.713213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.713481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.714117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.717245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.750002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1052/2116] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:14 2022] POST /identity/v3/projects => generated 359 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:14.755081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.769050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96eefe-c62f-4567-b721-66b25d281e92 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.833274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.833983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.834292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.834613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.835263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.837339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.866998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1065/2117] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:14 2022] POST /identity/v3/users => generated 280 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:14.870948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.881729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f21d-115d-440a-8cd1-cdf097ee5005 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.886434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.887514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.887795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.888066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.888736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.890495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.902070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1053/2118] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:14.908847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.920078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b71c-4571-4cb9-88bf-48ab257363a2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.925596 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.926279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.926560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.926834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.927480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.929145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.941975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1066/2119] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:14.946186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.963727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e09e47-954f-41e1-81ed-0f19b356c1ca admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.969426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c37a2-8edc-47da-a2ff-8c018c18fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.970542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c37a2-8edc-47da-a2ff-8c018c18fc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.971056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c37a2-8edc-47da-a2ff-8c018c18fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.971609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c37a2-8edc-47da-a2ff-8c018c18fc03 admin admin] PATH_INFO: `/v3/projects/c40a51fb2681486cad677ed307f29c80/users/d9a727a3539349ceb973be8eb098fa91/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.990722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.991879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c37a2-8edc-47da-a2ff-8c018c18fc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40a51fb2681486cad677ed307f29c80, user_id=d9a727a3539349ceb973be8eb098fa91, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.992846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.995125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c37a2-8edc-47da-a2ff-8c018c18fc03 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.007241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69132-5306-4707-86e5-fe1e7c0ddd82 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.010881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1054/2120] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:14 2022] PUT /identity/v3/projects/c40a51fb2681486cad677ed307f29c80/users/d9a727a3539349ceb973be8eb098fa91/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:15.022070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.022354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.022647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.051277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] MFA Rules not processed for user `d9a727a3539349ceb973be8eb098fa91`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:15.052628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.071817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.072474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.073006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.073157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] PATH_INFO: `/v3/users/f6e3ba7d542a4e19b65cf569b6dd398f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.074174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e3ba7d542a4e19b65cf569b6dd398f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.076854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.100443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.123182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1055/2121] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:15.133876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e3ba7d542a4e19b65cf569b6dd398f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.141864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e3ba7d542a4e19b65cf569b6dd398f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.155697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e3ba7d542a4e19b65cf569b6dd398f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.156573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e3ba7d542a4e19b65cf569b6dd398f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.158595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.159992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.168101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-eb9af1dd-d2e8-4949-a9b4-76c99650d726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e3ba7d542a4e19b65cf569b6dd398f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.174949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1067/2122] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:14 2022] DELETE /identity/v3/users/f6e3ba7d542a4e19b65cf569b6dd398f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:15.179378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.179688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.179961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.196799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.201941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:15.203443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.224382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc816b-0886-4776-92dc-1e4487782896 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.266183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1068/2123] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:15.276212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.277209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.277591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.278790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.279492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.279763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.281502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.353694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf6337-d1c6-4024-9ae9-21de829aa840 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.358096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.358687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.358959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.359710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.361755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.363223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.368393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.369877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.372890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1056/2124] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:15.373738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.375073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.388281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1069/2125] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:15.390055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.390539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee673f44-4b0c-46fc-9822-9815d27b9f64 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.391609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.397791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.398075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.398827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.398827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] PATH_INFO: `/v3/users/69e66e65f5eb4816a5e8fd5381a9e2a8` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.400064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e66e65f5eb4816a5e8fd5381a9e2a8)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.402140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.407974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdab5-458e-499c-ac81-a245df3832ea service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.418515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.419713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.419986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.420252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] PATH_INFO: `/v3/users/e9426f8949d54a6ab6aa6cc6114147d5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.422254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9426f8949d54a6ab6aa6cc6114147d5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.425438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.461712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e66e65f5eb4816a5e8fd5381a9e2a8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.472693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e66e65f5eb4816a5e8fd5381a9e2a8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.473591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e66e65f5eb4816a5e8fd5381a9e2a8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.481601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.485899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e66e65f5eb4816a5e8fd5381a9e2a8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.495076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9426f8949d54a6ab6aa6cc6114147d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.502093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e66e65f5eb4816a5e8fd5381a9e2a8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.506778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9426f8949d54a6ab6aa6cc6114147d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.510742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1057/2126] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:15 2022] DELETE /identity/v3/users/69e66e65f5eb4816a5e8fd5381a9e2a8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:15.515131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.516368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.523705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9426f8949d54a6ab6aa6cc6114147d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.524585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9426f8949d54a6ab6aa6cc6114147d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.532982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.535355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5073df09-0ed8-4e45-93ef-fc649fd7b2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9426f8949d54a6ab6aa6cc6114147d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.547468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1070/2127] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:15 2022] DELETE /identity/v3/users/e9426f8949d54a6ab6aa6cc6114147d5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:15.549449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10e76-6810-4a7c-ac59-542f1225ece3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.549639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10e76-6810-4a7c-ac59-542f1225ece3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.549868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10e76-6810-4a7c-ac59-542f1225ece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.583603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-01f10e76-6810-4a7c-ac59-542f1225ece3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:15.585274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-01f10e76-6810-4a7c-ac59-542f1225ece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.600933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a492f-5587-483d-a108-6ea7db584b14 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.664982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1071/2128] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:15.666499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.668498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.668863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.669139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.675072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.675544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.675855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] PATH_INFO: `/v3/users/d9a727a3539349ceb973be8eb098fa91` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.677322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a727a3539349ceb973be8eb098fa91)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.679417 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.692226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:15.693938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.703798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1072/2129] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:15.704999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.706538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.764500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a727a3539349ceb973be8eb098fa91'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.773905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a727a3539349ceb973be8eb098fa91'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.774714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a727a3539349ceb973be8eb098fa91'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.794656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a727a3539349ceb973be8eb098fa91'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.803549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a610bf-f3e3-4e14-9060-02db32ff3bd7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.807977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.809030 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.809918 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.810198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.810482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a727a3539349ceb973be8eb098fa91'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:15.814252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.826197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1058/2130] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:15 2022] DELETE /identity/v3/users/d9a727a3539349ceb973be8eb098fa91 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:15.829280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.829280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.835489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.837896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.853133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1073/2131] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:15.853991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.855358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:15.883575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1328a9d-b042-45d1-8d63-577b2ada4438 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.911696 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.958172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c940b41-7c3e-4b2a-b36b-e950616d37c3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.964470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.967257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.967546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.968198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] PATH_INFO: `/v3/projects/acb6cb6c88774f6bbee061efd7618719` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.969219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb6cb6c88774f6bbee061efd7618719)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.971877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.983814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.984497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.984807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.985067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.993006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.009363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:16.019895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.022062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1782b-fbe7-4d01-ad66-70180e3872e4 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.039438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1074/2132] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:16.041020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.041484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.041809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.048130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb6cb6c88774f6bbee061efd7618719', 'request_id': 'req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.056576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb6cb6c88774f6bbee061efd7618719', 'request_id': 'req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.068745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:16.070770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.079032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb6cb6c88774f6bbee061efd7618719', 'request_id': 'req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.097598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] The token cache is being invalidate because project acb6cb6c88774f6bbee061efd7618719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:16.097874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.099084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.100200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1059/2133] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:15 2022] DELETE /identity/v3/projects/acb6cb6c88774f6bbee061efd7618719 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:16.114259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.115959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.225665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a359d-72aa-454a-9288-b84dfc9ff8b4 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.226303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1075/2134] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:16.230779 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.231889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.231889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.232124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] PATH_INFO: `/v3/users/d2c4f2669d584c7eaaa8d27effd5b3c1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.233151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c4f2669d584c7eaaa8d27effd5b3c1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.235307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.238767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.242004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.293322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c4f2669d584c7eaaa8d27effd5b3c1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.302930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c4f2669d584c7eaaa8d27effd5b3c1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.303757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c4f2669d584c7eaaa8d27effd5b3c1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.313683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c4f2669d584c7eaaa8d27effd5b3c1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.321631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:16.329689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c4f2669d584c7eaaa8d27effd5b3c1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.332825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432c25f-70c8-43fc-ad52-c1d27fe43faa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.337514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.338336 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.338675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.339663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.342317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1060/2135] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:16 2022] DELETE /identity/v3/users/d2c4f2669d584c7eaaa8d27effd5b3c1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:16.344986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.359213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.360947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.389311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1076/2136] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:16.625339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.626861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.715008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0acbb-27bc-4296-af7f-dfd238a86739 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.723135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.724152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.724636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.725116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] PATH_INFO: `/v3/projects/0a371d5a13d54a01b879645b26a1ebe9` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.727484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a371d5a13d54a01b879645b26a1ebe9)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.729917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.827304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a371d5a13d54a01b879645b26a1ebe9', 'request_id': 'req-ead14a6e-9db0-4502-871f-8dd405b7da0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.827414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.838295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.859409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a371d5a13d54a01b879645b26a1ebe9', 'request_id': 'req-ead14a6e-9db0-4502-871f-8dd405b7da0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.870811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943501d9-858f-4150-a0aa-b491949e69b5 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.877836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a371d5a13d54a01b879645b26a1ebe9', 'request_id': 'req-ead14a6e-9db0-4502-871f-8dd405b7da0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.919452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] The token cache is being invalidate because project 0a371d5a13d54a01b879645b26a1ebe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:16.921735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.924221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:16.927952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1061/2137] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:16 2022] DELETE /identity/v3/projects/0a371d5a13d54a01b879645b26a1ebe9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:16.969841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.971403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.045146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.046463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.047005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.047480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] PATH_INFO: `/v3/projects/57a9714e40d94a2ab2fada6ad1f6184a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.049146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a9714e40d94a2ab2fada6ad1f6184a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.051422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.070569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14a6e-9db0-4502-871f-8dd405b7da0f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.076297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.077148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.077532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.077953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] PATH_INFO: `/v3/projects/e2739c58b1274d5aa2fc0b76e55cb905` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.079523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2739c58b1274d5aa2fc0b76e55cb905)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.081876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.116027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.128325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.141226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a9714e40d94a2ab2fada6ad1f6184a', 'request_id': 'req-470385ed-de95-4927-9216-589818ffe4cc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.159009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a9714e40d94a2ab2fada6ad1f6184a', 'request_id': 'req-470385ed-de95-4927-9216-589818ffe4cc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.161500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2739c58b1274d5aa2fc0b76e55cb905', 'request_id': 'req-79a04514-fffa-4c3a-bccc-2241dd9674f3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.171563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a9714e40d94a2ab2fada6ad1f6184a', 'request_id': 'req-470385ed-de95-4927-9216-589818ffe4cc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.185942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2739c58b1274d5aa2fc0b76e55cb905', 'request_id': 'req-79a04514-fffa-4c3a-bccc-2241dd9674f3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.193241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] The token cache is being invalidate because project 57a9714e40d94a2ab2fada6ad1f6184a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:17.193715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.194692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.195404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2739c58b1274d5aa2fc0b76e55cb905', 'request_id': 'req-79a04514-fffa-4c3a-bccc-2241dd9674f3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.200523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1077/2138] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:16 2022] DELETE /identity/v3/projects/57a9714e40d94a2ab2fada6ad1f6184a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:17.200523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.201512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.216835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] The token cache is being invalidate because project e2739c58b1274d5aa2fc0b76e55cb905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:17.217140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.217829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.219729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1062/2139] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:16 2022] DELETE /identity/v3/projects/e2739c58b1274d5aa2fc0b76e55cb905 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:17.223322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.223322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.300826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04514-fffa-4c3a-bccc-2241dd9674f3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.305958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.306675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.307429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.307429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] PATH_INFO: `/v3/users/8bc9b975ef8d4d6f906999d6f210812e` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.308616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc9b975ef8d4d6f906999d6f210812e)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.310919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.368333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470385ed-de95-4927-9216-589818ffe4cc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.375073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.376059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.376514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.376913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] PATH_INFO: `/v3/projects/c40a51fb2681486cad677ed307f29c80` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.381770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40a51fb2681486cad677ed307f29c80)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.383980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.386363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc9b975ef8d4d6f906999d6f210812e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.401760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc9b975ef8d4d6f906999d6f210812e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.404804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc9b975ef8d4d6f906999d6f210812e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.417317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc9b975ef8d4d6f906999d6f210812e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.436631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8dfd4a14-aa42-4559-a450-b309eff05c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc9b975ef8d4d6f906999d6f210812e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.444120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1063/2140] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:17 2022] DELETE /identity/v3/users/8bc9b975ef8d4d6f906999d6f210812e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:17.447091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4b62e-1568-40a2-bd86-b605f9bc641b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.447091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4b62e-1568-40a2-bd86-b605f9bc641b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.447091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4b62e-1568-40a2-bd86-b605f9bc641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.453110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40a51fb2681486cad677ed307f29c80', 'request_id': 'req-f447d157-b8e4-4176-abd4-d895fb71b658'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.460959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40a51fb2681486cad677ed307f29c80', 'request_id': 'req-f447d157-b8e4-4176-abd4-d895fb71b658'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.471660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40a51fb2681486cad677ed307f29c80', 'request_id': 'req-f447d157-b8e4-4176-abd4-d895fb71b658'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.477837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bfa4b62e-1568-40a2-bd86-b605f9bc641b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:17.479828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bfa4b62e-1568-40a2-bd86-b605f9bc641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.499531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] The token cache is being invalidate because project c40a51fb2681486cad677ed307f29c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:17.499833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.501649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f447d157-b8e4-4176-abd4-d895fb71b658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:17.505949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1078/2141] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:17 2022] DELETE /identity/v3/projects/c40a51fb2681486cad677ed307f29c80 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:17.514053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.514931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13461-e96b-4aa3-8e59-eca468da8788 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.515247 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13461-e96b-4aa3-8e59-eca468da8788 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.515767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13461-e96b-4aa3-8e59-eca468da8788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.561823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-44a13461-e96b-4aa3-8e59-eca468da8788 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:17.566238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-44a13461-e96b-4aa3-8e59-eca468da8788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.582241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1064/2142] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:17.583980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.584370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.584740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.611151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:17.613018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.627695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1065/2143] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:17.628877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.662431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.682283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10d06-e94b-4c51-89d5-8797df8c969a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.683179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1079/2144] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:17.687718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.688001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.688267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.689841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.690581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.690946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.691898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.692179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:17.692575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.695215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.715168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:17.717559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:17.728616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1066/2145] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:17.729564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.741573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1080/2146] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:17.747389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.751051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119f48a-02a9-4a9c-b30c-5f2a10d21729 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.758797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.759612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.760574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.761161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.762020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.765332 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.775491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb46220-5bbd-44f8-bc15-8a1967f78a8a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.794283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.794936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.795628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.795628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.798213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:17.798543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.800320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.804329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1067/2147] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:17.809628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.820925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1081/2148] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:17.822851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.849072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815bc1-11d2-490d-a85d-44e0901881cf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.941445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c887519-22d2-4367-9bd4-ed3580e5e2dc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.958964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.959896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.960256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.960606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.961355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.963276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.963657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.963968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.964388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.964662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.965345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.969104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.004747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1068/2149] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:18.005718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.009625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.020811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1082/2150] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:17 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:18.022163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.026137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92948e-1fa9-4c73-b75c-5c9e6dc247b7 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.117423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.118451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.119148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.122832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] PATH_INFO: `/v3/projects/c65c8317ba6b468e99c5f2df36f6f84b` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.124651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65c8317ba6b468e99c5f2df36f6f84b)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.128291 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.131262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae95ed-cbd4-4bdd-8df9-67e45b1ce75b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.155430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.156689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.157367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.157860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.158827 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.161397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.172123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.178801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1083/2151] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:18.181566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.195104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55028cd-057b-4ab7-ae9f-edaf89986710 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.205920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65c8317ba6b468e99c5f2df36f6f84b', 'request_id': 'req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:18.216956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65c8317ba6b468e99c5f2df36f6f84b', 'request_id': 'req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:18.238146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65c8317ba6b468e99c5f2df36f6f84b', 'request_id': 'req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:18.270147 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] The token cache is being invalidate because project c65c8317ba6b468e99c5f2df36f6f84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:18.270386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:18.273048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:18.277109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1069/2152] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:18 2022] DELETE /identity/v3/projects/c65c8317ba6b468e99c5f2df36f6f84b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:18.282011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.283397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.321117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53dd0e-740b-4b8f-b932-5f7b26b3d876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.321755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53dd0e-740b-4b8f-b932-5f7b26b3d876 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.322096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53dd0e-740b-4b8f-b932-5f7b26b3d876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.322382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53dd0e-740b-4b8f-b932-5f7b26b3d876 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.323165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53dd0e-740b-4b8f-b932-5f7b26b3d876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.326683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53dd0e-740b-4b8f-b932-5f7b26b3d876 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.336674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.367245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1084/2153] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:18.367245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.367245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.367245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.392886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:18.394352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.395190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801eba6-6ccc-4cb4-be90-3d045c4c21b7 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.410920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1085/2154] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:18.412200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.413806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.432634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.433406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.434920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.435775 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50896c-fae0-4bbd-be0c-8bc0b228699b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.436005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.443394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e24793-575e-4fc8-8702-58caadac4f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.444515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e24793-575e-4fc8-8702-58caadac4f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.445195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e24793-575e-4fc8-8702-58caadac4f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.445747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e24793-575e-4fc8-8702-58caadac4f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.446643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e24793-575e-4fc8-8702-58caadac4f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.446759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.448430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.448900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e24793-575e-4fc8-8702-58caadac4f2f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.467095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1086/2155] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:18.472409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.472699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.473026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.474438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1070/2156] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:18.475163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.487716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b643-7d3b-4096-adc3-9a29732906d8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.497908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.498886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.499375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.499802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.501905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.504538 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.515889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:18.517277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.528159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1071/2157] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:18.529579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.533905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1087/2158] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:18.534758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.549105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caeb20-67d6-42d5-b711-912152554d45 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.550480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b0d25-55a6-4be0-84ac-de481e618e4e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.554939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd77b1d-a267-42d4-be7e-541cd279b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.555973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77b1d-a267-42d4-be7e-541cd279b51b admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.556988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77b1d-a267-42d4-be7e-541cd279b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.558341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.558496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd77b1d-a267-42d4-be7e-541cd279b51b admin admin] PATH_INFO: `/v3/projects/11c86f60c4ae4ce181cd278a29003d2a/users/d24e6805905d4fff8bd522a8a676601b/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.559315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.561027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.561404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] PATH_INFO: `/v3/projects/614c447d527e451ea97c2d919e59247e/users/2429b8b9019d4d619ea49a546272dad7/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.586723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=614c447d527e451ea97c2d919e59247e, user_id=2429b8b9019d4d619ea49a546272dad7, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.589592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.597765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd77b1d-a267-42d4-be7e-541cd279b51b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c86f60c4ae4ce181cd278a29003d2a, user_id=d24e6805905d4fff8bd522a8a676601b, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.601005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd77b1d-a267-42d4-be7e-541cd279b51b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.620814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1088/2159] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:18 2022] PUT /identity/v3/projects/614c447d527e451ea97c2d919e59247e/users/2429b8b9019d4d619ea49a546272dad7/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:18.620814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.626212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1072/2160] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:18 2022] PUT /identity/v3/projects/11c86f60c4ae4ce181cd278a29003d2a/users/d24e6805905d4fff8bd522a8a676601b/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:18.628144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0dad6-2d9a-4f87-903e-6764fe7dca40 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.628470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0dad6-2d9a-4f87-903e-6764fe7dca40 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.628880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0dad6-2d9a-4f87-903e-6764fe7dca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.666667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-afe0dad6-2d9a-4f87-903e-6764fe7dca40 None None] MFA Rules not processed for user `2429b8b9019d4d619ea49a546272dad7`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:18.667985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-afe0dad6-2d9a-4f87-903e-6764fe7dca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.710598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae9a42-0004-456c-a2cb-8ac9f90531fd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.744717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.758706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1073/2161] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:18.760152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.760444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.760748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.787885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:18.789145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.806485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.808389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.808915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.810445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.811170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:18.811478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.813340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.826621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1089/2162] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:18.827482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.829514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.863867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1074/2163] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:18.865464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.897577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.911854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e4af4-dea4-43d2-873c-10aab4e1fb2b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.920371 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd41a23-26ac-4945-afa4-5143f3113d8c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.925862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.927365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.928021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.928299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.928592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.929678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.929979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.930265 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.931038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.936430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.941419 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.942523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.964321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1090/2164] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:18.967496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.970362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.979443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1075/2165] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:18.980419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.047263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ad400-fcc9-4d61-9b25-d894b4316836 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.067096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc408d85-a8be-4d38-991e-f21a15fd2c01 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.072495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.073128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.073399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.073661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.074695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.077299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.102963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1076/2166] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:19.103887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.105765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.109867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.110579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.110979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.111323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.119551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d86587-f0cc-45a0-99b2-7adb6cd4c769 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.121465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.124206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.125287 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.126051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.126348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.126749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] PATH_INFO: `/v3/users/53730ff461b142208d249a93842934f1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.128415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] RBAC: Authorizing `identity:delete_user(user_id=53730ff461b142208d249a93842934f1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.130428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.139599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1091/2167] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:19.140384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.151655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e990-be02-4c46-91ae-22e0f93ec377 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.157592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.158418 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.158702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.159378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.159603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.161537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.174973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1092/2168] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:19.179686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.197605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23013cb8-29cd-47b0-8ad0-9d6745cc681d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.201435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53730ff461b142208d249a93842934f1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:19.203981 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.204562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.204946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.205367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.206016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.207643 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.217030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53730ff461b142208d249a93842934f1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:19.217775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53730ff461b142208d249a93842934f1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:19.221652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1093/2169] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:19.223710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.225755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.227840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.235493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53730ff461b142208d249a93842934f1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:19.243189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0aa5-8dcc-4c5c-a21e-52c6b82c0ce5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.250228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854b9c-85f8-4aad-9d6c-a30178a763ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.250901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854b9c-85f8-4aad-9d6c-a30178a763ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.251160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854b9c-85f8-4aad-9d6c-a30178a763ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.251439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854b9c-85f8-4aad-9d6c-a30178a763ec admin admin] PATH_INFO: `/v3/projects/cf4d0ef13e1b447f88f62042546bcc7a/users/3dfb9c17f9a943b895b89fe1e1403de4/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.259344 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d173a17a-541f-4698-a6fe-be1a44a003de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53730ff461b142208d249a93842934f1'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:19.267543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1077/2170] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:19 2022] DELETE /identity/v3/users/53730ff461b142208d249a93842934f1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:19.271995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.272951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.274161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.278509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854b9c-85f8-4aad-9d6c-a30178a763ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4d0ef13e1b447f88f62042546bcc7a, user_id=3dfb9c17f9a943b895b89fe1e1403de4, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.281696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854b9c-85f8-4aad-9d6c-a30178a763ec admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.319551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1094/2171] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:19 2022] PUT /identity/v3/projects/cf4d0ef13e1b447f88f62042546bcc7a/users/3dfb9c17f9a943b895b89fe1e1403de4/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:19.332077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:19.333677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.338923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.339886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.341770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.408409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] MFA Rules not processed for user `3dfb9c17f9a943b895b89fe1e1403de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:19.410061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.476241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.477159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1078/2172] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:19.489855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.491400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.514278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1095/2173] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:19.527373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.529915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.569765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f146b3-2b89-482b-8f38-dc0f285afd51 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.577464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fbc3e-21cc-4160-90d1-4917ff5023e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.578183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fbc3e-21cc-4160-90d1-4917ff5023e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.578473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fbc3e-21cc-4160-90d1-4917ff5023e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.578750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fbc3e-21cc-4160-90d1-4917ff5023e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.602992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fbc3e-21cc-4160-90d1-4917ff5023e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.604460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fbc3e-21cc-4160-90d1-4917ff5023e9 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.619479 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1079/2174] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:19.641797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e7464-19be-4a7f-9f6d-caeb9b550733 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.646535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.651199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.651658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.651919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.665234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.666695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.678577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1096/2175] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:20.260155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.260594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.261005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.288615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.290897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:20.312256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] MFA Rules not processed for user `d24e6805905d4fff8bd522a8a676601b`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:20.313664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:20.314486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11303aba-878f-493c-ae08-2406c6732280 service glance] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.321441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.322308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.322911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.323294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] PATH_INFO: `/v3/projects/cfe796b4da6748b6b44eaf9cd42d98ac` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.325053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe796b4da6748b6b44eaf9cd42d98ac)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.327350 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.424220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe796b4da6748b6b44eaf9cd42d98ac', 'request_id': 'req-be645a09-37ae-4db6-aa93-cea0b5191a3d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.436690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1080/2176] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:20.441255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.443521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:20.443878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe796b4da6748b6b44eaf9cd42d98ac', 'request_id': 'req-be645a09-37ae-4db6-aa93-cea0b5191a3d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.480847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe796b4da6748b6b44eaf9cd42d98ac', 'request_id': 'req-be645a09-37ae-4db6-aa93-cea0b5191a3d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.482346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79524ec3-af4b-4784-a8f4-eceb9dc506c0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.506196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:20.510287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] The token cache is being invalidate because project cfe796b4da6748b6b44eaf9cd42d98ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:20.510611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.511305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.516262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1097/2177] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:20 2022] DELETE /identity/v3/projects/cfe796b4da6748b6b44eaf9cd42d98ac => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:20.518920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.520248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:20.596940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.597759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.598032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.598310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.598943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.601051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.614219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:20.630079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be645a09-37ae-4db6-aa93-cea0b5191a3d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.636347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1081/2178] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:20 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:20.639362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.640013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.640295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.641403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.641708 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] PATH_INFO: `/v3/users/7d579cd2393c48ceafb50dd2e3508298` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.643582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d579cd2393c48ceafb50dd2e3508298)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.643706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:20.645157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.717219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d579cd2393c48ceafb50dd2e3508298'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.729618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d579cd2393c48ceafb50dd2e3508298'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.730124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bcf78-6caa-4782-a6e7-472e739ff972 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.739239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.739855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.740129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.740398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.741055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.743036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.746053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d579cd2393c48ceafb50dd2e3508298'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.747097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d579cd2393c48ceafb50dd2e3508298'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.755150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d579cd2393c48ceafb50dd2e3508298'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:20.763143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1098/2179] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:20 2022] DELETE /identity/v3/users/7d579cd2393c48ceafb50dd2e3508298 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:20.770886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1082/2180] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:20 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:20.774915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.776385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:20.859469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf94ad-d4f4-47d5-8b66-744e2e1e492b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.864302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.865020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.865376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.865716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.866438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.868210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.878974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1099/2181] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:20.883346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.894774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e4dcf-13d0-4696-9a9d-009ce744ad0d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.899639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.900288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.900579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.900927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] PATH_INFO: `/v3/projects/0a23dd62c7eb4eda80fb63bd32ba4eb1/users/8eb88c5b7d2c49439020299a17f38c84/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.921165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a23dd62c7eb4eda80fb63bd32ba4eb1, user_id=8eb88c5b7d2c49439020299a17f38c84, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.924408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.941611 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1083/2182] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:20 2022] PUT /identity/v3/projects/0a23dd62c7eb4eda80fb63bd32ba4eb1/users/8eb88c5b7d2c49439020299a17f38c84/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:20.945694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.957567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a060a-b902-41bc-9a75-b17ceab93d81 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.056512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.057180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.057576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.057845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.058454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.060549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.066774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:21.069028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:21.081077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1100/2183] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:21.082007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:21.091462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:21.098128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16efc5d-fef7-4454-b2b0-06b627bbed77 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.102951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e802c77-0b21-4878-bf00-1956686cf52d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.104730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.105578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.105844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.106124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] PATH_INFO: `/v3/projects/0a23dd62c7eb4eda80fb63bd32ba4eb1/users/8eb88c5b7d2c49439020299a17f38c84/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.111728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.112502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.112887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.113095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a23dd62c7eb4eda80fb63bd32ba4eb1, user_id=8eb88c5b7d2c49439020299a17f38c84, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.113237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] PATH_INFO: `/v3/projects/7bed4dc04a29497a89c587ffe2ba34ee` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.114653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bed4dc04a29497a89c587ffe2ba34ee)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.117314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.117729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.135789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1101/2184] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:21 2022] PUT /identity/v3/projects/0a23dd62c7eb4eda80fb63bd32ba4eb1/users/8eb88c5b7d2c49439020299a17f38c84/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:21.186698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bed4dc04a29497a89c587ffe2ba34ee', 'request_id': 'req-5bbfea73-f3d9-42d0-9051-e0450bbf279d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.194091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bed4dc04a29497a89c587ffe2ba34ee', 'request_id': 'req-5bbfea73-f3d9-42d0-9051-e0450bbf279d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.196880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:21.204602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bed4dc04a29497a89c587ffe2ba34ee', 'request_id': 'req-5bbfea73-f3d9-42d0-9051-e0450bbf279d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.219992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] The token cache is being invalidate because project 7bed4dc04a29497a89c587ffe2ba34ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:21.220369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.221245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.222912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1084/2185] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:21 2022] DELETE /identity/v3/projects/7bed4dc04a29497a89c587ffe2ba34ee => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:21.227060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:21.228890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:21.309271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65926ef-c8e2-4bb2-92c1-c21f9577759f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.404610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.405274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.405719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.406250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] PATH_INFO: `/v3/users/64c21cbee3c343a2a0d613d20d3ff1c9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.408134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c21cbee3c343a2a0d613d20d3ff1c9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.411855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.505270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c21cbee3c343a2a0d613d20d3ff1c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.519477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c21cbee3c343a2a0d613d20d3ff1c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.549946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c21cbee3c343a2a0d613d20d3ff1c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.550911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c21cbee3c343a2a0d613d20d3ff1c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.563689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2f8990f4-d4e0-440a-8a63-606b029acdf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c21cbee3c343a2a0d613d20d3ff1c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:21.570786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1102/2186] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:21 2022] DELETE /identity/v3/users/64c21cbee3c343a2a0d613d20d3ff1c9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:22.229915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.241499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfea73-f3d9-42d0-9051-e0450bbf279d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.321091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.321747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.322022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.322313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] PATH_INFO: `/v3/projects/8ba11eae063e421891d7a2c844750709` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.323469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba11eae063e421891d7a2c844750709)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.326049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.385328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba11eae063e421891d7a2c844750709', 'request_id': 'req-a29f4e94-62a7-4958-ac7b-8abf338eac4f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:22.392505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba11eae063e421891d7a2c844750709', 'request_id': 'req-a29f4e94-62a7-4958-ac7b-8abf338eac4f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:22.401704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba11eae063e421891d7a2c844750709', 'request_id': 'req-a29f4e94-62a7-4958-ac7b-8abf338eac4f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:22.417588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] The token cache is being invalidate because project 8ba11eae063e421891d7a2c844750709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:22.417816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:22.418562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a29f4e94-62a7-4958-ac7b-8abf338eac4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:22.421958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1085/2187] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:22 2022] DELETE /identity/v3/projects/8ba11eae063e421891d7a2c844750709 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:22.426964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f66c7-8e95-4a1b-a6e9-9024192e06cd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.428001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f66c7-8e95-4a1b-a6e9-9024192e06cd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.428287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f66c7-8e95-4a1b-a6e9-9024192e06cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.454698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5a7f66c7-8e95-4a1b-a6e9-9024192e06cd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:22.458282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5a7f66c7-8e95-4a1b-a6e9-9024192e06cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.480389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.484519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.484969 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.492085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:22.535520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] MFA Rules not processed for user `8eb88c5b7d2c49439020299a17f38c84`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:22.538454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.634752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1103/2188] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:22.642835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.643404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.643709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.666895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:22.668129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.680805 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1104/2189] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:22.684323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.689121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:22.694841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1086/2190] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:22.700068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b1ca9-b344-44bd-bfc7-933c3f4d3b66 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.706151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.706914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.707229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.707605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.708622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:22.708750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.709905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.710514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.713257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.726299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1105/2191] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:22.730576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.746821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7f2e1-bd20-46e5-8bf2-853dbdac80fe admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.756338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.757005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.757288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.757562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.758245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.760174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.802513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f557-176c-48a9-a36f-0518739079f4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.806176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1106/2192] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:22.807060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.871170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.872074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.872512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.873039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.874914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.886403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcf9a2-b200-4830-a1d7-5026405c1524 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.893916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.895343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.895343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.895343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.896014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.898255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.930899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1107/2193] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:22.935818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.953517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764fcbb-5cdd-4813-8fc9-14d15752e15a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.961594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.962251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.962533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.962813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.963516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.965196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.965680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.967937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.981152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1108/2194] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:22.981373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1087/2195] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:22.984321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.995344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8347-d79d-4c3f-bc19-8941f371eb12 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.000416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be494c0-11c7-405e-92f1-facf09e4ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.001216 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be494c0-11c7-405e-92f1-facf09e4ae83 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.001563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be494c0-11c7-405e-92f1-facf09e4ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.001907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be494c0-11c7-405e-92f1-facf09e4ae83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.003029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be494c0-11c7-405e-92f1-facf09e4ae83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.004969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be494c0-11c7-405e-92f1-facf09e4ae83 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.020921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1109/2196] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:23.025028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.037718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b353-d5cd-42aa-8573-6a94b3b5507f service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.042587 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.043269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.043755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.044482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] PATH_INFO: `/v3/projects/f82c6dd205d640a4a78c23050a8f73af/users/ed0a2e4acb9f4eab98f96e3fa3001099/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.065127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82c6dd205d640a4a78c23050a8f73af, user_id=ed0a2e4acb9f4eab98f96e3fa3001099, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.069077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.093051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1088/2197] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:23 2022] PUT /identity/v3/projects/f82c6dd205d640a4a78c23050a8f73af/users/ed0a2e4acb9f4eab98f96e3fa3001099/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:23.098260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a9d97-d171-4d38-9ace-5f0aabbfddbd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.098549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a9d97-d171-4d38-9ace-5f0aabbfddbd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.098824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a9d97-d171-4d38-9ace-5f0aabbfddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.122433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ea6a9d97-d171-4d38-9ace-5f0aabbfddbd None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:23.124106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ea6a9d97-d171-4d38-9ace-5f0aabbfddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:23.184337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.225249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1110/2198] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:23.240148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.243775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:23.380053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.408285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e613a4-caca-4da8-8ad2-7b3bfbbd1973 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.414559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.415516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.416434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.416716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.430833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.432685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.453591 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1089/2199] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:24.195093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.197022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.199573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.252431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] MFA Rules not processed for user `ed0a2e4acb9f4eab98f96e3fa3001099`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:24.254365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:24.352121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1111/2200] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:24.361543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.438729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef78edf-4efa-40fb-85eb-00778ee80281 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.455193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.455193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.455193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.455597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.463565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.465107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:24.477929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.480141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.483033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbc32a-5db9-4237-a33c-a08311498828 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.488622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96ca58-f9b6-4ef2-9b6b-6aeda9f86a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.489542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96ca58-f9b6-4ef2-9b6b-6aeda9f86a48 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.490001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96ca58-f9b6-4ef2-9b6b-6aeda9f86a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.490447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96ca58-f9b6-4ef2-9b6b-6aeda9f86a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.492983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1090/2201] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:24.591742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96ca58-f9b6-4ef2-9b6b-6aeda9f86a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.593502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96ca58-f9b6-4ef2-9b6b-6aeda9f86a48 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.609515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1112/2202] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:26.046065 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.047512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:26.070416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cffad-b597-4397-af71-5beaf127ac4f service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.081518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.081518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.081518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.081518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] PATH_INFO: `/v3/users/ce4bf5ffbf2d47c9b54c4c1e343e8944` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.081958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce4bf5ffbf2d47c9b54c4c1e343e8944)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.083224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.104646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.105252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.105252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.127499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] MFA Rules not processed for user `ed0a2e4acb9f4eab98f96e3fa3001099`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:26.128956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:26.136585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1113/2203] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:26.149476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce4bf5ffbf2d47c9b54c4c1e343e8944'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.158618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce4bf5ffbf2d47c9b54c4c1e343e8944'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.159374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce4bf5ffbf2d47c9b54c4c1e343e8944'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.175762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce4bf5ffbf2d47c9b54c4c1e343e8944'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.197371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-20c75bee-540f-403b-9c19-4672dad692e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce4bf5ffbf2d47c9b54c4c1e343e8944'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.207518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1091/2204] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:26 2022] DELETE /identity/v3/users/ce4bf5ffbf2d47c9b54c4c1e343e8944 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:26.460688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.479650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162e91e-d4b6-4ea3-9d9a-16b1a54f9d5d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.557584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.558315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.558669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.559014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] PATH_INFO: `/v3/projects/0be0cbafba614105869a0913a21a127b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.561056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be0cbafba614105869a0913a21a127b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.563224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.571841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:26.623992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be0cbafba614105869a0913a21a127b', 'request_id': 'req-81b3c51e-2e07-4989-9631-3ea283c461d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.631120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be0cbafba614105869a0913a21a127b', 'request_id': 'req-81b3c51e-2e07-4989-9631-3ea283c461d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.640083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be0cbafba614105869a0913a21a127b', 'request_id': 'req-81b3c51e-2e07-4989-9631-3ea283c461d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.651077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] The token cache is being invalidate because project 0be0cbafba614105869a0913a21a127b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:26.651368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.652126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81b3c51e-2e07-4989-9631-3ea283c461d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:26.653732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1114/2205] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:26 2022] DELETE /identity/v3/projects/0be0cbafba614105869a0913a21a127b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:26.658762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.659138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.659478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.679220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:26.680912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:26.748029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1092/2206] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:26.754288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.754580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.754860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.775111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:26.776422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:26.784473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1115/2207] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:26.788966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.796167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:26.800911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e61f-8aaa-4510-b68a-c1fdc743ceb2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.807512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.808218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.808502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.808793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.809526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:26.810721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.812658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.827592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1093/2208] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:26.832048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.843642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de9a6-9665-4fe4-9cc0-112d79453658 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.848331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.849025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.849251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.849521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.850157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.852095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.908535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.915405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1116/2209] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:26 2022] POST /identity/v3/projects => generated 389 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:26.925517 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3588e3f-4448-47a1-bf72-cee4217ee1db admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.000059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.001184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.001581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.002007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.003309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:27.005884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:27.032392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1094/2210] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:26 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:27.033780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.044031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7013-463a-4c42-9cf0-2dfd11d7902f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.049411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.049995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.050265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.050528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.051160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:27.052801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:27.066672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1117/2211] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:27.067983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.079500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920e6dc-5a66-4ae7-b6e3-64cad5cdf58e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.084158 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae843195-c929-4b7e-a628-c2482d2064c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.084914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae843195-c929-4b7e-a628-c2482d2064c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.085193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae843195-c929-4b7e-a628-c2482d2064c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.085462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae843195-c929-4b7e-a628-c2482d2064c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.086316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae843195-c929-4b7e-a628-c2482d2064c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:27.087951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae843195-c929-4b7e-a628-c2482d2064c0 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:27.099275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1095/2212] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:27.103523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.115551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40126-900c-4353-85f0-2c41e4e33af8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.120218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.120982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.121323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.121659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] PATH_INFO: `/v3/projects/b5da9e76bed44f3d90144e4fbff2cb1c/users/1af20de2433c44ff8fb1f16290404c21/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.138594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5da9e76bed44f3d90144e4fbff2cb1c, user_id=1af20de2433c44ff8fb1f16290404c21, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:27.141573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:27.157153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1118/2213] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:27 2022] PUT /identity/v3/projects/b5da9e76bed44f3d90144e4fbff2cb1c/users/1af20de2433c44ff8fb1f16290404c21/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:27.162408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc829ae-142a-407f-8199-4f3afff71921 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.162689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc829ae-142a-407f-8199-4f3afff71921 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.163034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc829ae-142a-407f-8199-4f3afff71921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.183551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2cc829ae-142a-407f-8199-4f3afff71921 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:27.184781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2cc829ae-142a-407f-8199-4f3afff71921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:27.258621 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1096/2214] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:27.269648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.271381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:27.328957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b6bd-2213-4d1e-a0c7-2bd2ec482b64 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.332536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.333172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.333450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.333722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.343353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:27.344815 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:27.354105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1119/2215] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:32.678335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49246b-b0dc-4904-9097-6eae2179548a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.679091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49246b-b0dc-4904-9097-6eae2179548a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.679091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49246b-b0dc-4904-9097-6eae2179548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.712190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3d49246b-b0dc-4904-9097-6eae2179548a None None] MFA Rules not processed for user `1af20de2433c44ff8fb1f16290404c21`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:32.712190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3d49246b-b0dc-4904-9097-6eae2179548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:32.797844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1097/2216] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:32.798518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.811791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.813478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:32.854704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.949141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10e7d6-22cc-4edf-9205-af2316737c95 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.955618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53142376-7000-4202-8264-f6624aad63ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.956169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53142376-7000-4202-8264-f6624aad63ba service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.956450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53142376-7000-4202-8264-f6624aad63ba service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.956724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53142376-7000-4202-8264-f6624aad63ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.965103 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53142376-7000-4202-8264-f6624aad63ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:32.967283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53142376-7000-4202-8264-f6624aad63ba service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:32.976679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1120/2217] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:35.126618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5272f-1b9b-4977-8e90-da69175706f9 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:35.127771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5272f-1b9b-4977-8e90-da69175706f9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:35.128469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5272f-1b9b-4977-8e90-da69175706f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:35.149279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9cc5272f-1b9b-4977-8e90-da69175706f9 None None] MFA Rules not processed for user `1af20de2433c44ff8fb1f16290404c21`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:35.150522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9cc5272f-1b9b-4977-8e90-da69175706f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:35.160383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1098/2218] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:42.013762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53142376-7000-4202-8264-f6624aad63ba service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:42.015889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-53142376-7000-4202-8264-f6624aad63ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:42.029405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53142376-7000-4202-8264-f6624aad63ba service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:42.035657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:42.036875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.037392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.037736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] PATH_INFO: `/v3/users/2429b8b9019d4d619ea49a546272dad7` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.039007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2429b8b9019d4d619ea49a546272dad7)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:42.041151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:42.091749 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2429b8b9019d4d619ea49a546272dad7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.101033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2429b8b9019d4d619ea49a546272dad7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.115309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2429b8b9019d4d619ea49a546272dad7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.116143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2429b8b9019d4d619ea49a546272dad7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.125220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2429b8b9019d4d619ea49a546272dad7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.131641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1121/2219] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:42 2022] DELETE /identity/v3/users/2429b8b9019d4d619ea49a546272dad7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:42.136585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.137077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.137245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.159706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:42.160961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:42.250825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1099/2220] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:42.270112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:42.270112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:42.336539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474aaec-fc39-4c9e-ab4b-2d0afa8b19c5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:42.340663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:42.341346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.341792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.342101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.353777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:42.355227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:42.366058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1122/2221] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:42.755271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:42.756995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:42.778754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a829f24-73f4-4ceb-9907-b352c5212663 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:42.786094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:42.787235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.787672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.787943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] PATH_INFO: `/v3/projects/614c447d527e451ea97c2d919e59247e` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.789172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614c447d527e451ea97c2d919e59247e)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:42.793250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:42.857529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614c447d527e451ea97c2d919e59247e', 'request_id': 'req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.866397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614c447d527e451ea97c2d919e59247e', 'request_id': 'req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.877316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614c447d527e451ea97c2d919e59247e', 'request_id': 'req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.910426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] The token cache is being invalidate because project 614c447d527e451ea97c2d919e59247e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:42.910733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.911911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:42.915340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1100/2222] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:42 2022] DELETE /identity/v3/projects/614c447d527e451ea97c2d919e59247e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:44.085260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.086897 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:44.100554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a9b8-fd07-4579-81cc-485c0138501d service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.131062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.161014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.161644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.161925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.162282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] PATH_INFO: `/v3/users/d24e6805905d4fff8bd522a8a676601b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.163600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d24e6805905d4fff8bd522a8a676601b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.165302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.226340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd24e6805905d4fff8bd522a8a676601b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.238786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd24e6805905d4fff8bd522a8a676601b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.253437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd24e6805905d4fff8bd522a8a676601b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.254160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd24e6805905d4fff8bd522a8a676601b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.261593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd24e6805905d4fff8bd522a8a676601b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.268927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1123/2223] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:44 2022] DELETE /identity/v3/users/d24e6805905d4fff8bd522a8a676601b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:44.444441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.446098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:44.462215 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1ef5a-7ec2-49a8-a5cd-a90cdb9877f2 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.519385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.520004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.520285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.520552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] PATH_INFO: `/v3/projects/11c86f60c4ae4ce181cd278a29003d2a` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.521693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c86f60c4ae4ce181cd278a29003d2a)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.523609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.544363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.573139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c86f60c4ae4ce181cd278a29003d2a', 'request_id': 'req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.580177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c86f60c4ae4ce181cd278a29003d2a', 'request_id': 'req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.588057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c86f60c4ae4ce181cd278a29003d2a', 'request_id': 'req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.599233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] The token cache is being invalidate because project 11c86f60c4ae4ce181cd278a29003d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:44.599520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.600152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.601666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1101/2224] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:44 2022] DELETE /identity/v3/projects/11c86f60c4ae4ce181cd278a29003d2a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:44.605669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.607081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:44.620987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241c681-f496-4c1a-ada6-6f55712df4fb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.678313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.678933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.679210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.679479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] PATH_INFO: `/v3/users/8eb88c5b7d2c49439020299a17f38c84` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.680590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb88c5b7d2c49439020299a17f38c84)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.682584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.729318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb88c5b7d2c49439020299a17f38c84'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.732992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.737967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb88c5b7d2c49439020299a17f38c84'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.752658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb88c5b7d2c49439020299a17f38c84'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.753329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb88c5b7d2c49439020299a17f38c84'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.760372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d0ea0586-93ce-4f66-8abc-ee3f3190f63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb88c5b7d2c49439020299a17f38c84'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:44.767482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1124/2225] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:44 2022] DELETE /identity/v3/users/8eb88c5b7d2c49439020299a17f38c84 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:44.974962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.976442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:44.992870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c184f2f-6c10-46a3-aa2f-7d4bfd797353 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.084024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.085537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.085822 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.086159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] PATH_INFO: `/v3/projects/0a23dd62c7eb4eda80fb63bd32ba4eb1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.087253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a23dd62c7eb4eda80fb63bd32ba4eb1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.089246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.139612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a23dd62c7eb4eda80fb63bd32ba4eb1', 'request_id': 'req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:45.146492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a23dd62c7eb4eda80fb63bd32ba4eb1', 'request_id': 'req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:45.161579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a23dd62c7eb4eda80fb63bd32ba4eb1', 'request_id': 'req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:45.172807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] The token cache is being invalidate because project 0a23dd62c7eb4eda80fb63bd32ba4eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:45.173094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:45.173733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4ca0ca67-0614-4bd6-afb8-d19f1a3b6077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:45.175189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1102/2226] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:44 2022] DELETE /identity/v3/projects/0a23dd62c7eb4eda80fb63bd32ba4eb1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:45.179946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.180392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.180522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.204450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:45.205709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:45.279872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1125/2227] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:45.289955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.290238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.290513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.311556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:45.312745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:45.320563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1103/2228] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:45.324755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.334658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34a235-604c-409e-94d1-9420b91c9648 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.339485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.340079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.340358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.340633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.341316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:45.341622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.343734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.355577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1126/2229] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:45.361600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.370983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b187f9-d8e9-44d9-9f6d-3cf13275193a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.376126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.376726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.377040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.377366 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.378012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.380192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.412199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1104/2230] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:45.417219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.429547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496fdbd-e8e8-480a-88f9-dcbe719dd58d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.489589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.490215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.490568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.490916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.491821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.494012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.519745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1127/2231] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:45.521141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.531796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e572e8f-ef53-4791-ad8f-e22a5adf670b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.536408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.537027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.537310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.537585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.538224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.539833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.541926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:45.552623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1105/2232] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:45.556308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.567256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce32928-a4ea-4f57-8ea2-938c784cee9d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.573019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2ae7b-a52a-427d-953d-a3a12cfc5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.574028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2ae7b-a52a-427d-953d-a3a12cfc5fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.574501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2ae7b-a52a-427d-953d-a3a12cfc5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.574962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2ae7b-a52a-427d-953d-a3a12cfc5fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.576070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2ae7b-a52a-427d-953d-a3a12cfc5fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.578997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2ae7b-a52a-427d-953d-a3a12cfc5fc1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.591389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1128/2233] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:45.595812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.605765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68241f28-fd06-4a8e-aa29-fe41eeeb6241 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.611155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.611782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.612058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.612326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] PATH_INFO: `/v3/projects/9a578b375db442e8be1bcb88c37a1c3a/users/633b97d18c2e411e90719d513aac6bbf/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.627251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a578b375db442e8be1bcb88c37a1c3a, user_id=633b97d18c2e411e90719d513aac6bbf, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.630204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.642784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1106/2234] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:45 2022] PUT /identity/v3/projects/9a578b375db442e8be1bcb88c37a1c3a/users/633b97d18c2e411e90719d513aac6bbf/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:45.653016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fd946-983e-427b-9442-b544241d0693 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.653303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fd946-983e-427b-9442-b544241d0693 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.653576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fd946-983e-427b-9442-b544241d0693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.681369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-888fd946-983e-427b-9442-b544241d0693 None None] MFA Rules not processed for user `633b97d18c2e411e90719d513aac6bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:45.682577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-888fd946-983e-427b-9442-b544241d0693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:45.706689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:45.752294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1129/2235] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:45.756894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.819614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862dd15-2d13-4e3e-b56d-9bf92bb4c747 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.824275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.824952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.825233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.825502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.826118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.828384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.855676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1107/2236] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:45.860376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fd946-983e-427b-9442-b544241d0693 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.871196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fd946-983e-427b-9442-b544241d0693 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.930258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.930853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.931130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.931397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.932006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.933979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.955595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1130/2237] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:45.959842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.971008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb336d8-c687-4a11-9cba-a114703278d9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.975655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.976246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.976528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.976830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.977560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.979238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.991161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1108/2238] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:45.995232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.006082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28e5bd-f340-499d-af47-9d619093608a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.010536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.011163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.011437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.011706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] PATH_INFO: `/v3/projects/f7d31a48d4334f448a1c6643186a8764/users/3ae214b158444bd8b0ae577e3c4c7580/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.026578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d31a48d4334f448a1c6643186a8764, user_id=3ae214b158444bd8b0ae577e3c4c7580, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.029322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.043331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1131/2239] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:45 2022] PUT /identity/v3/projects/f7d31a48d4334f448a1c6643186a8764/users/3ae214b158444bd8b0ae577e3c4c7580/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:46.047980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.058227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55023c-0dfb-4ad4-9be2-60ec1c26f7cc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.116849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7a7cd-3d46-454e-bb78-cd43ab718990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.117483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7a7cd-3d46-454e-bb78-cd43ab718990 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.117760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7a7cd-3d46-454e-bb78-cd43ab718990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.118058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7a7cd-3d46-454e-bb78-cd43ab718990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.118765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7a7cd-3d46-454e-bb78-cd43ab718990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.120431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7a7cd-3d46-454e-bb78-cd43ab718990 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.130972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1109/2240] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:46.135091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.145601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4526-fcdb-440e-aee4-8d4c08ffd8f4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.150434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.151081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.151362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.151640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] PATH_INFO: `/v3/projects/f7d31a48d4334f448a1c6643186a8764/users/3ae214b158444bd8b0ae577e3c4c7580/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.153593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d31a48d4334f448a1c6643186a8764, user_id=3ae214b158444bd8b0ae577e3c4c7580, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.156847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.167862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1132/2241] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:46 2022] PUT /identity/v3/projects/f7d31a48d4334f448a1c6643186a8764/users/3ae214b158444bd8b0ae577e3c4c7580/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:46.179689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.180092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.180250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.212706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] MFA Rules not processed for user `3ae214b158444bd8b0ae577e3c4c7580`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:46.214203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:46.255561 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.295522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1110/2242] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:46.307090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.309524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:46.378326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc7bc8-9903-4a34-a919-ab1b8b1ed19a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.382910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293eced-a69a-4a02-b06f-c1a016cf7c69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.383512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293eced-a69a-4a02-b06f-c1a016cf7c69 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.383791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293eced-a69a-4a02-b06f-c1a016cf7c69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.384056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293eced-a69a-4a02-b06f-c1a016cf7c69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.422644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.463398 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293eced-a69a-4a02-b06f-c1a016cf7c69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.464898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293eced-a69a-4a02-b06f-c1a016cf7c69 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.476591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1133/2243] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:47.794099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.795616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:47.867158 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f27b7-38d1-4feb-b40b-c58ad62181cc None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.872575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.873492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.875242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.876270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] PATH_INFO: `/v3/users/3dfb9c17f9a943b895b89fe1e1403de4` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.877760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfb9c17f9a943b895b89fe1e1403de4)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.880943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.955922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfb9c17f9a943b895b89fe1e1403de4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:47.973416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfb9c17f9a943b895b89fe1e1403de4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:47.974315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfb9c17f9a943b895b89fe1e1403de4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:47.989247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfb9c17f9a943b895b89fe1e1403de4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.004539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfb9c17f9a943b895b89fe1e1403de4'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.012328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1111/2244] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:47 2022] DELETE /identity/v3/users/3dfb9c17f9a943b895b89fe1e1403de4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:48.017138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.017588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.017863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.043443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:48.044720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:48.113479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1134/2245] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:48.125968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:48.127467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:48.182745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:48.186555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b9fad-2106-4e9a-b9a2-328fa21f8c6f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:48.191422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9edb8-e336-4bd3-bd52-81a334507086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:48.192034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9edb8-e336-4bd3-bd52-81a334507086 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.192524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9edb8-e336-4bd3-bd52-81a334507086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.193127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9edb8-e336-4bd3-bd52-81a334507086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.202454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9edb8-e336-4bd3-bd52-81a334507086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:48.203931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9edb8-e336-4bd3-bd52-81a334507086 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:48.213475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1112/2246] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:48.492965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:48.494562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:48.516379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de842aa-28cd-4ef3-8aba-8de7eb0c9f3c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:48.521852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:48.522601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.522976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.523318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] PATH_INFO: `/v3/projects/cf4d0ef13e1b447f88f62042546bcc7a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.524989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4d0ef13e1b447f88f62042546bcc7a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:48.527654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:48.562881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed7c32-15e1-4159-ba57-b37f5f4ccd8a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.563164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed7c32-15e1-4159-ba57-b37f5f4ccd8a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.563444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed7c32-15e1-4159-ba57-b37f5f4ccd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.586511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-96ed7c32-15e1-4159-ba57-b37f5f4ccd8a None None] MFA Rules not processed for user `ed0a2e4acb9f4eab98f96e3fa3001099`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:48.588007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-96ed7c32-15e1-4159-ba57-b37f5f4ccd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:48.619106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4d0ef13e1b447f88f62042546bcc7a', 'request_id': 'req-475ecc9d-2024-4e62-a60b-ec5048feb3aa'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.628700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4d0ef13e1b447f88f62042546bcc7a', 'request_id': 'req-475ecc9d-2024-4e62-a60b-ec5048feb3aa'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.637725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4d0ef13e1b447f88f62042546bcc7a', 'request_id': 'req-475ecc9d-2024-4e62-a60b-ec5048feb3aa'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.653288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1113/2247] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:48.657155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] The token cache is being invalidate because project cf4d0ef13e1b447f88f62042546bcc7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:48.657447 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.658205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-475ecc9d-2024-4e62-a60b-ec5048feb3aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:48.660342 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1135/2248] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:48 2022] DELETE /identity/v3/projects/cf4d0ef13e1b447f88f62042546bcc7a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:48.674353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.674717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.675089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.695613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:48.697083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:48.779648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1114/2249] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:48.788871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.789211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.789490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.815284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:48.816537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:48.825051 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1136/2250] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:48.831103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:48.847464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61787a9-2805-4efc-af19-36239c44a321 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:48.853061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:48.853854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.854761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.854761 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.855618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:48.857565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:48.860647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:48.877199 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1115/2251] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:48.884295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:48.896450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca908557-38a1-4bd3-96ac-5226a2669d25 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:48.903235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:48.903870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:48.904153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:48.904424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.910594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:48.910594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:48.958129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:48.969061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1137/2252] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:48 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:48.975977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d872c20-6c47-4f1d-814e-a25e504d0f74 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.064614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.065295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.065585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.065861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.066523 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.069025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.110968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1116/2253] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:48 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:49.115686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.128933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a7355-7b5d-4c65-be18-8218404e2ecb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.137599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.138559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.139039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.139488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.140422 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.142509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.156960 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1138/2254] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:49.160911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.176478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40166-8ed0-48c6-9f89-acba7bb2c195 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.185796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.186242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.187016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.187718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.188452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.189211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.189391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.191197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.203647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1117/2255] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:49.204174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca18b1-ae67-4fb4-8bd1-f61442a02cb4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.208725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.213312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.213975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.214414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.214693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] PATH_INFO: `/v3/users/633b97d18c2e411e90719d513aac6bbf` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.215392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:49.216963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] RBAC: Authorizing `identity:delete_user(user_id=633b97d18c2e411e90719d513aac6bbf)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.221640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.225275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f279c9-938f-4e05-a777-537d4d834d76 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.234427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c911b-215b-42a0-81c4-2f4562c7407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.237031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c911b-215b-42a0-81c4-2f4562c7407d admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.237331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c911b-215b-42a0-81c4-2f4562c7407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.237604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c911b-215b-42a0-81c4-2f4562c7407d admin admin] PATH_INFO: `/v3/projects/81a4648acb3b4f33a98dc58536c9d855/users/3d3a0e0cbda0427f839751f786bea809/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.243779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:49.296802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c911b-215b-42a0-81c4-2f4562c7407d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a4648acb3b4f33a98dc58536c9d855, user_id=3d3a0e0cbda0427f839751f786bea809, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.296802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c911b-215b-42a0-81c4-2f4562c7407d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.307236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633b97d18c2e411e90719d513aac6bbf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:49.311108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1118/2256] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:49 2022] PUT /identity/v3/projects/81a4648acb3b4f33a98dc58536c9d855/users/3d3a0e0cbda0427f839751f786bea809/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:49.317023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633b97d18c2e411e90719d513aac6bbf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:49.324240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.324964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.325388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.333697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633b97d18c2e411e90719d513aac6bbf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:49.334520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633b97d18c2e411e90719d513aac6bbf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:49.342391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a9ef3a7-edee-4877-a6fd-c9262bd4944d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633b97d18c2e411e90719d513aac6bbf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:49.353971 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1139/2257] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:49 2022] DELETE /identity/v3/users/633b97d18c2e411e90719d513aac6bbf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:49.356431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.356852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.357386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.367680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] MFA Rules not processed for user `3d3a0e0cbda0427f839751f786bea809`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:49.369515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.385305 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:49.387635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.485290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1140/2258] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:49.494307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1119/2259] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:49.501537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.503276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.513431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.515809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.589283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7938ca-8e45-4adb-94d3-573ee61618bf None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.593324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab7621-7649-48d2-863c-9fb43a0c9279 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.594735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.596913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.597195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.597470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.598191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.598893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.599400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.599710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.610273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.610850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:49.611747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.616791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:49.623063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1141/2260] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:49.630484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1120/2261] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:50.078547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.080353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:50.115784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ece11-a6e1-4ba3-a4f8-70dfc6ba949e service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.120466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.121113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.121394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.121662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] PATH_INFO: `/v3/projects/9a578b375db442e8be1bcb88c37a1c3a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.123044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a578b375db442e8be1bcb88c37a1c3a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.125310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.164934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.185986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a578b375db442e8be1bcb88c37a1c3a', 'request_id': 'req-07033582-5c2f-4770-aeb4-52898141b19d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.193704 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a578b375db442e8be1bcb88c37a1c3a', 'request_id': 'req-07033582-5c2f-4770-aeb4-52898141b19d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.217638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a578b375db442e8be1bcb88c37a1c3a', 'request_id': 'req-07033582-5c2f-4770-aeb4-52898141b19d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.233864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] The token cache is being invalidate because project 9a578b375db442e8be1bcb88c37a1c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:50.234195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.235458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.237478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1142/2262] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:50 2022] DELETE /identity/v3/projects/9a578b375db442e8be1bcb88c37a1c3a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:50.241470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.253990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a40b-669d-47de-b646-9ca4a9849671 service glance] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.316700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.317387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.317670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.317975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] PATH_INFO: `/v3/users/3ae214b158444bd8b0ae577e3c4c7580` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.319120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae214b158444bd8b0ae577e3c4c7580)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.321074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.334550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.383275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae214b158444bd8b0ae577e3c4c7580'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.393323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae214b158444bd8b0ae577e3c4c7580'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.394359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae214b158444bd8b0ae577e3c4c7580'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.403604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae214b158444bd8b0ae577e3c4c7580'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.420519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-fbde41de-e71d-4dcf-a70a-b5dad6e7e82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae214b158444bd8b0ae577e3c4c7580'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:50.432007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1121/2263] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:31:50 2022] DELETE /identity/v3/users/3ae214b158444bd8b0ae577e3c4c7580 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:50.956959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.958673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:50.975137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07033582-5c2f-4770-aeb4-52898141b19d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.056244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.057001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.057347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.057686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] PATH_INFO: `/v3/projects/f7d31a48d4334f448a1c6643186a8764` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.058998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d31a48d4334f448a1c6643186a8764)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.062302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.124006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d31a48d4334f448a1c6643186a8764', 'request_id': 'req-ec4668f5-f96f-43d2-8d7f-b3959514845b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:51.131446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d31a48d4334f448a1c6643186a8764', 'request_id': 'req-ec4668f5-f96f-43d2-8d7f-b3959514845b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:51.139634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d31a48d4334f448a1c6643186a8764', 'request_id': 'req-ec4668f5-f96f-43d2-8d7f-b3959514845b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:51.153995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] The token cache is being invalidate because project f7d31a48d4334f448a1c6643186a8764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:31:51.154582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:51.155260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ec4668f5-f96f-43d2-8d7f-b3959514845b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:31:51.157493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1143/2264] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:31:50 2022] DELETE /identity/v3/projects/f7d31a48d4334f448a1c6643186a8764 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:51.163279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.163703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.164078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.184135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:51.186076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:51.262545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1122/2265] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:51.270312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.270802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.271222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.292758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:51.294025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:51.308389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1144/2266] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:51.310056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.332748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79fbe2-0de1-476c-96e3-7f311e6adbfa None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.337367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.337973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.338252 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.338525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.339193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:51.339475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.341151 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.361333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1123/2267] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:51.362779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.372475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458be2ab-5343-4be4-93b0-18b50aea0032 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.378905 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.379816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.380464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.380875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.381588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.383532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.419681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1145/2268] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:31:51 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:51.424633 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.435643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e5f7-3392-42b8-ac10-cf99388b0e4e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.497509 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.498179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.498462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.498736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.499413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.501397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.526837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1124/2269] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:51 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:51.531546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.542294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa458b0-6c45-413c-a431-a02710bbd2dd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.547936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.548658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.549587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.549974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.550744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.552449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.567451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1146/2270] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:51.574749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.586554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24502ed7-e384-4ab9-bdc0-1060ea8f8dbc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.591470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580cb16-958f-489a-989f-39f9d622bfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.592109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580cb16-958f-489a-989f-39f9d622bfaf admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.592394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580cb16-958f-489a-989f-39f9d622bfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.592666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580cb16-958f-489a-989f-39f9d622bfaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.593919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580cb16-958f-489a-989f-39f9d622bfaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.595653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580cb16-958f-489a-989f-39f9d622bfaf admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.597213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.613861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1125/2271] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:31:51.614554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.621355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.626494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931703-3b0f-4523-8867-03339854da44 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.633876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.634965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.635442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.635982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] PATH_INFO: `/v3/projects/9ffd7ab0d575409a897b9a2414043dbc/users/04ef1cc341de4dc9b5de95a6c0b3c9f4/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.656504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffd7ab0d575409a897b9a2414043dbc, user_id=04ef1cc341de4dc9b5de95a6c0b3c9f4, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.659295 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.674464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1147/2272] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:31:51 2022] PUT /identity/v3/projects/9ffd7ab0d575409a897b9a2414043dbc/users/04ef1cc341de4dc9b5de95a6c0b3c9f4/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:51.685034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887cf5f-46e1-480f-bfc7-4125dab2cd9d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.685357 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887cf5f-46e1-480f-bfc7-4125dab2cd9d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.685590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887cf5f-46e1-480f-bfc7-4125dab2cd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.715538 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f887cf5f-46e1-480f-bfc7-4125dab2cd9d None None] MFA Rules not processed for user `04ef1cc341de4dc9b5de95a6c0b3c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:51.716812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f887cf5f-46e1-480f-bfc7-4125dab2cd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:51.810599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1126/2273] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:51.822893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.824681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:51.898217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2e85-3c67-4e1e-9d05-416928c73883 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.902117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.902729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.903010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.903317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.913504 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.916124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.926814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1148/2274] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:01.807351 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=34 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:01.922610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:12.518998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb51f-6f2c-4368-a407-53796cdea564 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:12.519510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb51f-6f2c-4368-a407-53796cdea564 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:12.519642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb51f-6f2c-4368-a407-53796cdea564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:12.540047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ce4fb51f-6f2c-4368-a407-53796cdea564 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:12.541474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ce4fb51f-6f2c-4368-a407-53796cdea564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:12.600925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1127/2275] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:32:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:12.644497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:12.646156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:12.716444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6ed7f-003a-4e99-b3ba-bcc48c5c6d26 service glance] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:12.721550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546c28b-687a-4318-9f86-cbc691af6749 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:12.722166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546c28b-687a-4318-9f86-cbc691af6749 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:12.722430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546c28b-687a-4318-9f86-cbc691af6749 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:12.722700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546c28b-687a-4318-9f86-cbc691af6749 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:12.732707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546c28b-687a-4318-9f86-cbc691af6749 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:12.734906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546c28b-687a-4318-9f86-cbc691af6749 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:12.742953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1149/2276] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:32:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:32:22.594184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=25 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:22.739310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=32 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:27.480997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:27.481513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:27.481682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:27.526617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] MFA Rules not processed for user `33b932f3feb342d889155c3c34407a88`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:27.529240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:27.644919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1128/2277] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:35.056182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.056804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.057087 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:35.093912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:35.095278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:35.121049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1150/2278] 10.222.0.81 () {58 vars in 967 bytes} [Wed Aug 24 03:32:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:35.134196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:35.151397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fb421-922b-4ae6-8bc1-764b957e2f6c None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:35.157806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:35.158129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.158401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.158705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:35.169838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:35.171495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:35.198528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1129/2279] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:32:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:32:37.544128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:37.545635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:37.613757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4f083-b4c8-4dfe-834c-3927f4840346 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:37.618483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:37.619099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:37.619373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:37.619642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] PATH_INFO: `/v3/users/ed0a2e4acb9f4eab98f96e3fa3001099` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:37.620789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0a2e4acb9f4eab98f96e3fa3001099)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:37.623010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:37.673601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0a2e4acb9f4eab98f96e3fa3001099'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:37.681518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0a2e4acb9f4eab98f96e3fa3001099'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:37.694895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0a2e4acb9f4eab98f96e3fa3001099'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:37.695668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0a2e4acb9f4eab98f96e3fa3001099'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:37.706995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10ea1ce5-e6ec-444b-81ed-2fea09684b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0a2e4acb9f4eab98f96e3fa3001099'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:37.714344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1151/2280] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:32:37 2022] DELETE /identity/v3/users/ed0a2e4acb9f4eab98f96e3fa3001099 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:37.962604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:37.976951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109485c-ef67-4089-8c16-ef34bcea0055 service swift] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.064026 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.064654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.065516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.066109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] PATH_INFO: `/v3/projects/f82c6dd205d640a4a78c23050a8f73af` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.067535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82c6dd205d640a4a78c23050a8f73af)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.069653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.122942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82c6dd205d640a4a78c23050a8f73af', 'request_id': 'req-270a4e67-c635-429d-ac4d-0263549695e9'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:38.129834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82c6dd205d640a4a78c23050a8f73af', 'request_id': 'req-270a4e67-c635-429d-ac4d-0263549695e9'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:38.136624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82c6dd205d640a4a78c23050a8f73af', 'request_id': 'req-270a4e67-c635-429d-ac4d-0263549695e9'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:38.147245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] The token cache is being invalidate because project f82c6dd205d640a4a78c23050a8f73af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:32:38.147535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:38.148171 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-270a4e67-c635-429d-ac4d-0263549695e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:38.149728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1130/2281] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:32:37 2022] DELETE /identity/v3/projects/f82c6dd205d640a4a78c23050a8f73af => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:38.154682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.154970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.155269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.176106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:38.177344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:38.241537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1152/2282] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:38.249482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.249768 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.250163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.270529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:38.271831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:38.279791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1131/2283] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:38.284952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.294825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dafb8-12dd-470e-b86b-24c3598b967d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.300215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.300743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.301052 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.301324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.301964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:38.302263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.303861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.312928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:38.316625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1153/2284] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:38.321407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.330507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e79966-b779-4d7e-ad9d-a4bab544ed23 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.338052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.338819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.339096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.339365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.340019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.341954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.357513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:38.371837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1132/2285] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:32:38 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:38.376015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.386106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31281-c4a4-47e0-a0a0-0d2948833bea admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.447729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.448990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.449268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.449531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.450151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.452147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.473933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1154/2286] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:32:38 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:32:38.478242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.489499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3244-886c-4638-93a9-eb00cbbf9393 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.494692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.495572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.495858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.496131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.496846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.498460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.509107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1133/2287] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:38.513147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.523850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a4fc2-6d44-48f9-a77f-c85e3e8a7ea6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.528467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a7332-ab51-4901-8415-f210af56111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.529090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a7332-ab51-4901-8415-f210af56111c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.529441 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a7332-ab51-4901-8415-f210af56111c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.529888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a7332-ab51-4901-8415-f210af56111c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.530721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a7332-ab51-4901-8415-f210af56111c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.532756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a7332-ab51-4901-8415-f210af56111c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.553628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1155/2288] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:38.556050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.568632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bef142-f442-4ed7-83e7-9cfd939464b3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.578427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.579642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.580107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.580986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] PATH_INFO: `/v3/projects/f48cbfe0a91142ea89379ac62ae091f6/users/600cda2d233e461a95635079217b172e/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.599297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48cbfe0a91142ea89379ac62ae091f6, user_id=600cda2d233e461a95635079217b172e, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.602032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.622680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1134/2289] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:38 2022] PUT /identity/v3/projects/f48cbfe0a91142ea89379ac62ae091f6/users/600cda2d233e461a95635079217b172e/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:38.633608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.634273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.634673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.668222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] MFA Rules not processed for user `600cda2d233e461a95635079217b172e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:38.669638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:38.745618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1156/2290] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:38.757838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:38.759539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:38.824743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73214df4-49b1-4c6e-aa1a-fed85f083a3e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:38.828485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15100ef3-f5b8-445f-b6a6-4c023f84671e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:38.829122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15100ef3-f5b8-445f-b6a6-4c023f84671e service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:38.829396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15100ef3-f5b8-445f-b6a6-4c023f84671e service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:38.829664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15100ef3-f5b8-445f-b6a6-4c023f84671e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:38.839865 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15100ef3-f5b8-445f-b6a6-4c023f84671e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:38.841420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15100ef3-f5b8-445f-b6a6-4c023f84671e service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:38.852027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1135/2291] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:39.974654 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:39.976380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.044292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc673ac-806a-4438-8a6d-75d0c7970bc9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.048909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.049597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.049878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.050151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] PATH_INFO: `/v3/users/04ef1cc341de4dc9b5de95a6c0b3c9f4` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.051277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ef1cc341de4dc9b5de95a6c0b3c9f4)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.053254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.064043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:40.100679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ef1cc341de4dc9b5de95a6c0b3c9f4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.107829 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ef1cc341de4dc9b5de95a6c0b3c9f4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.120672 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ef1cc341de4dc9b5de95a6c0b3c9f4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.121358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ef1cc341de4dc9b5de95a6c0b3c9f4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.128392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ef1cc341de4dc9b5de95a6c0b3c9f4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.135572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1157/2292] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:32:39 2022] DELETE /identity/v3/users/04ef1cc341de4dc9b5de95a6c0b3c9f4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:40.139952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.140330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.140500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.161300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:40.162520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.223232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1136/2293] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:40.233168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.234713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.316275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.318137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.319387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d362fe-372c-4610-bf27-7d9b8dcdb924 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.324075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace9881-c6fd-468d-bc95-bcfcbdb78e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.324987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace9881-c6fd-468d-bc95-bcfcbdb78e73 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.325558 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace9881-c6fd-468d-bc95-bcfcbdb78e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.325898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace9881-c6fd-468d-bc95-bcfcbdb78e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.337395 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c524-d7c3-43a7-b503-7bb6d3be700b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.341324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace9881-c6fd-468d-bc95-bcfcbdb78e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.342196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.342808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace9881-c6fd-468d-bc95-bcfcbdb78e73 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.342934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.343112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.343380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] PATH_INFO: `/v3/users/3d3a0e0cbda0427f839751f786bea809` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.344626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3a0e0cbda0427f839751f786bea809)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.348228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.357369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1158/2294] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:40.413410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3a0e0cbda0427f839751f786bea809'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.422439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3a0e0cbda0427f839751f786bea809'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.423241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3a0e0cbda0427f839751f786bea809'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.428211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:40.435348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3a0e0cbda0427f839751f786bea809'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.460368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3a0e0cbda0427f839751f786bea809'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.471860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1137/2295] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:32:40 2022] DELETE /identity/v3/users/3d3a0e0cbda0427f839751f786bea809 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:40.479728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.480298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.480821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.506425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:40.507787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.577917 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1159/2296] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:40.587594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.589177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.649929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5d318-9250-4e9b-94b3-9be5e6bc4b9c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.654059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92657fad-f2ff-42b2-b02d-29f5273aec75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.656134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.656753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92657fad-f2ff-42b2-b02d-29f5273aec75 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.657061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92657fad-f2ff-42b2-b02d-29f5273aec75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.657352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92657fad-f2ff-42b2-b02d-29f5273aec75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.658792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.671350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92657fad-f2ff-42b2-b02d-29f5273aec75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.671526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da943947-99e7-494a-99a7-7c8e4116934a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.672955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92657fad-f2ff-42b2-b02d-29f5273aec75 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.677317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.678153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.678608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.679089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] PATH_INFO: `/v3/projects/9ffd7ab0d575409a897b9a2414043dbc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.681065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffd7ab0d575409a897b9a2414043dbc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.682983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.690124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1138/2297] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:32:40.719088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:40.738688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffd7ab0d575409a897b9a2414043dbc', 'request_id': 'req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.745514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffd7ab0d575409a897b9a2414043dbc', 'request_id': 'req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.753713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffd7ab0d575409a897b9a2414043dbc', 'request_id': 'req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.765492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] The token cache is being invalidate because project 9ffd7ab0d575409a897b9a2414043dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:32:40.765787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.766436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-04dfc92f-0650-4cc5-8770-68bcb1aaa56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:40.768106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1160/2298] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:32:40 2022] DELETE /identity/v3/projects/9ffd7ab0d575409a897b9a2414043dbc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:40.774112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.774406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.774737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.797532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:40.799375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.867952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1139/2299] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:40.875274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.875779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.876197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.901909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:40.903146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:40.919746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1161/2300] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:40.925294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.935565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83ffa9-1db7-42df-8896-e032bcc0636d None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.941027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.941641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.941914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.942183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.942837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:40.943194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.944970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.958551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1140/2301] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:40.962736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.972851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43f294-15fb-4678-a30c-2b6aa4d0af8b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.977991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.978634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.978879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.979152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.979784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.981791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.016321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.018922 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.020076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1162/2302] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:32:40 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:41.024412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.041503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a961df-0de5-480c-b669-f9ff616bf247 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.103129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360143d3-1746-443e-817b-ce7a1e0fde2f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.108612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.109276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.109683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.109961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.110589 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.112569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.114618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.115427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.115884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.116303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] PATH_INFO: `/v3/projects/81a4648acb3b4f33a98dc58536c9d855` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.118742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a4648acb3b4f33a98dc58536c9d855)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.122876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.131432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:41.139171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1163/2303] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:41.144603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.156463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08b73-09cb-4503-a9d5-fb6d0d22f7a6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.162113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.162812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.163170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.163498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.164804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.166068 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.177513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1164/2304] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:32:41.181302 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.186792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a4648acb3b4f33a98dc58536c9d855', 'request_id': 'req-3d174609-84a5-4813-b817-51d2753dc50a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.191876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021fca2-805b-4dda-a6a5-af86409b5303 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.197608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a4648acb3b4f33a98dc58536c9d855', 'request_id': 'req-3d174609-84a5-4813-b817-51d2753dc50a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.208094 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a4648acb3b4f33a98dc58536c9d855', 'request_id': 'req-3d174609-84a5-4813-b817-51d2753dc50a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.221583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] The token cache is being invalidate because project 81a4648acb3b4f33a98dc58536c9d855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:32:41.221942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.222619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3d174609-84a5-4813-b817-51d2753dc50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.228430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1141/2305] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:32:41 2022] DELETE /identity/v3/projects/81a4648acb3b4f33a98dc58536c9d855 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:32:41.230299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68b56f-d382-440c-987e-04dcee54a305 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.230583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68b56f-d382-440c-987e-04dcee54a305 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.230853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68b56f-d382-440c-987e-04dcee54a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.252600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bd68b56f-d382-440c-987e-04dcee54a305 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:41.253913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bd68b56f-d382-440c-987e-04dcee54a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.262910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.263308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.263581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.263893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.264491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.266312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.272706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:41.281015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1165/2306] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:32:41.282025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.284363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.296964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6135fd-8f82-4910-a443-74acc70d7fe3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.303807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a2147-8946-46b6-a46a-496d01e1c3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.304541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a2147-8946-46b6-a46a-496d01e1c3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.305102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a2147-8946-46b6-a46a-496d01e1c3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.305219 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a2147-8946-46b6-a46a-496d01e1c3dc admin admin] PATH_INFO: `/v3/projects/0f92297ee0bb42da8eaace75baaab026/users/3d7250d93e9541438a6dfc4b87130b16/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.321792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a2147-8946-46b6-a46a-496d01e1c3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f92297ee0bb42da8eaace75baaab026, user_id=3d7250d93e9541438a6dfc4b87130b16, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.324507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a2147-8946-46b6-a46a-496d01e1c3dc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.337906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1142/2307] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:41.339629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1166/2308] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:41 2022] PUT /identity/v3/projects/0f92297ee0bb42da8eaace75baaab026/users/3d7250d93e9541438a6dfc4b87130b16/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:41.346588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.346891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.347313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.350991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.351693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.351693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.372350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:41.374294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.382120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] MFA Rules not processed for user `3d7250d93e9541438a6dfc4b87130b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:41.383718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.443875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1143/2309] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:41.448371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.458771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776146d-08a3-4ce0-ada8-911dc3e26289 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.468218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.468432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1167/2310] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:32:41.468521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.468521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.468521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.468521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:41.468521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.469403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.477354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.479319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.483981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1144/2311] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:41.488798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.499491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80699-394f-4de5-b678-54cb805cc618 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.506186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.506186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.506186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.506474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.507329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.509495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.538393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:41.546504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1145/2312] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:32:41.547464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.551246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.566647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4002b-18c0-436a-bd85-46e7004a09bb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.619606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.620135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.620413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.620684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.624070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107dc7b8-27fe-4c35-81ce-71b35519b6a7 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.630971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.631820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.632270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.633052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] PATH_INFO: `/v3/users/600cda2d233e461a95635079217b172e` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.634432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600cda2d233e461a95635079217b172e)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.636448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.671608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:41.694193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600cda2d233e461a95635079217b172e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.700077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.701620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.702424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600cda2d233e461a95635079217b172e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.703282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600cda2d233e461a95635079217b172e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.712509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1168/2313] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:32:41.713557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.714251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600cda2d233e461a95635079217b172e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.726311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a658ce-f7ff-43c4-8506-697b9a765d1c service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.730617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a542ac-7d80-429f-b59e-f6a39092daa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.731414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a542ac-7d80-429f-b59e-f6a39092daa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.731670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a542ac-7d80-429f-b59e-f6a39092daa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.731850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a542ac-7d80-429f-b59e-f6a39092daa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.732738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a542ac-7d80-429f-b59e-f6a39092daa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.734411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a542ac-7d80-429f-b59e-f6a39092daa3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.735329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600cda2d233e461a95635079217b172e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.748296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1146/2314] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:32:41 2022] DELETE /identity/v3/users/600cda2d233e461a95635079217b172e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:41.748970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.750411 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.763226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1169/2315] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:41.763226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.763226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.763226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.781631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:41.782913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.821429 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b19ed-171c-4466-b3de-9ce4857dd791 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.826449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.827086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.827367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.827638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] PATH_INFO: `/v3/users/3d7250d93e9541438a6dfc4b87130b16` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.829075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7250d93e9541438a6dfc4b87130b16)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.832154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.846251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1170/2316] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:41.847336 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.848687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.858382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:41.860590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc1991-c3a8-4d18-a240-8bd8fcf1217e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:41.865846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:41.866716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:41.866877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:41.867128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:41.867851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:41.869500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:41.881309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1171/2317] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:41.882198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.883980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.888531 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7250d93e9541438a6dfc4b87130b16'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.901396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7250d93e9541438a6dfc4b87130b16'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.902125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7250d93e9541438a6dfc4b87130b16'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.909820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7250d93e9541438a6dfc4b87130b16'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.924267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7250d93e9541438a6dfc4b87130b16'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:41.930740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1147/2318] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:32:41 2022] DELETE /identity/v3/users/3d7250d93e9541438a6dfc4b87130b16 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:41.931327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:41.934607 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:41.949804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca224e1-21e8-4424-bd0e-4a440431ff28 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.000961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b46f-48ab-45e5-a3c1-cf32f683ea15 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.001200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.002387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.002667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.002936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.004757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.006236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb29bb9-c6fa-4825-a6ef-519df2e10678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.006834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb29bb9-c6fa-4825-a6ef-519df2e10678 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.007108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb29bb9-c6fa-4825-a6ef-519df2e10678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.007376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb29bb9-c6fa-4825-a6ef-519df2e10678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.008013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb29bb9-c6fa-4825-a6ef-519df2e10678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.009634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb29bb9-c6fa-4825-a6ef-519df2e10678 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.017650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.019133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.019996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1148/2319] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:42.023219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.023219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.023219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.029983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1172/2320] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:42.030842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.040737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4ae49-c391-4f9f-88fd-aaca3f518220 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.040991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:42.042027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.045563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a67b7e-c6cc-4aa9-b647-844b8f3b8b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.046204 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a67b7e-c6cc-4aa9-b647-844b8f3b8b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.046472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a67b7e-c6cc-4aa9-b647-844b8f3b8b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.046742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a67b7e-c6cc-4aa9-b647-844b8f3b8b64 admin admin] PATH_INFO: `/v3/projects/60ed59a87fd44afb9064c8eef04b19c1/users/37b485ad806647f595c5b359505f1dd0/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.049566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:42.050402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1149/2321] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:42.063058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.065666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a67b7e-c6cc-4aa9-b647-844b8f3b8b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ed59a87fd44afb9064c8eef04b19c1, user_id=37b485ad806647f595c5b359505f1dd0, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.066257 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.068930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a67b7e-c6cc-4aa9-b647-844b8f3b8b64 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.080396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e49d5-01bc-4d2b-9f4c-381062d5dea9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.086880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1173/2322] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:42 2022] PUT /identity/v3/projects/60ed59a87fd44afb9064c8eef04b19c1/users/37b485ad806647f595c5b359505f1dd0/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:42.091854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.092140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.092420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.116196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:42.117702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.140222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.140935 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.141078 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.141333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.184406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1174/2323] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:42.193177 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:42.196180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.197925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.214814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be8d4-8ec6-46bd-b358-189c352a5847 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.218222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.219400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73051d-1172-45ec-b725-9ea4b0cc37d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.219670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.220132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73051d-1172-45ec-b725-9ea4b0cc37d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.220904 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73051d-1172-45ec-b725-9ea4b0cc37d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.221167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73051d-1172-45ec-b725-9ea4b0cc37d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.234471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73051d-1172-45ec-b725-9ea4b0cc37d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.234680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1150/2324] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:32:42.235951 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73051d-1172-45ec-b725-9ea4b0cc37d8 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.246272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1175/2325] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:32:42.384249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.386740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.404813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5038d-94b8-4de2-8065-5c81d489bde8 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.411355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.412361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.412720 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.413046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] PATH_INFO: `/v3/projects/f48cbfe0a91142ea89379ac62ae091f6` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.415208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48cbfe0a91142ea89379ac62ae091f6)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.418077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.486369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48cbfe0a91142ea89379ac62ae091f6', 'request_id': 'req-a346c43f-1f70-4dca-b80b-263fdc01a52d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.495363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48cbfe0a91142ea89379ac62ae091f6', 'request_id': 'req-a346c43f-1f70-4dca-b80b-263fdc01a52d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.505716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48cbfe0a91142ea89379ac62ae091f6', 'request_id': 'req-a346c43f-1f70-4dca-b80b-263fdc01a52d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.522778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] The token cache is being invalidate because project f48cbfe0a91142ea89379ac62ae091f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:32:42.523240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.524019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a346c43f-1f70-4dca-b80b-263fdc01a52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.525803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1151/2326] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:32:42 2022] DELETE /identity/v3/projects/f48cbfe0a91142ea89379ac62ae091f6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:42.530559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.531245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.531245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.558273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:42.559499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.628840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1176/2327] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:32:42.637409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.637570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.638052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.666471 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:42.667850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.677868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.679547 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.695374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1152/2328] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:42.700784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.702728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2e101-37e0-4213-b3e4-eb09030e83d1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.714997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.715939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.716982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.717400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] PATH_INFO: `/v3/projects/0f92297ee0bb42da8eaace75baaab026` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.719461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f92297ee0bb42da8eaace75baaab026)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.724723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.731908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8793fef-af87-4c9c-9cba-a0c917b9c223 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.741194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.744075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.744378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.744658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.745574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:42.745964 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.747763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.766250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1153/2329] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:42.768711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.783493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2144b60-46f9-4e61-9c07-add6b5abbb68 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:42.795040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f92297ee0bb42da8eaace75baaab026', 'request_id': 'req-ac1f6298-5dd8-414b-860b-fc0ce863fa81'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.802583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f92297ee0bb42da8eaace75baaab026', 'request_id': 'req-ac1f6298-5dd8-414b-860b-fc0ce863fa81'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.811086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f92297ee0bb42da8eaace75baaab026', 'request_id': 'req-ac1f6298-5dd8-414b-860b-fc0ce863fa81'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.830329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:42.839507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] The token cache is being invalidate because project 0f92297ee0bb42da8eaace75baaab026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:32:42.840089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.841140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:42.844153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1177/2330] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:32:42 2022] DELETE /identity/v3/projects/0f92297ee0bb42da8eaace75baaab026 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:42.871697 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:42.872326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.872602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.872914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.873755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:42.875725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:42.925966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1154/2331] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:32:42.926680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.930149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.951455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f6298-5dd8-414b-860b-fc0ce863fa81 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.033927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.034791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.035316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.035316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.040872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.042991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.075972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1178/2332] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:43.082005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.104057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605bd3-746b-4b3d-bc96-69ac74393738 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.111422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.112636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.113170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.113693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.115059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.117167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.131113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1155/2333] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:43.135234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.147750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115ea17-c72f-431b-aa03-d89b66043dd2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.152388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f5cf8-7d18-48fe-8e71-ade52554fe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.153127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f5cf8-7d18-48fe-8e71-ade52554fe89 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.153901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f5cf8-7d18-48fe-8e71-ade52554fe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.153901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f5cf8-7d18-48fe-8e71-ade52554fe89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.155139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f5cf8-7d18-48fe-8e71-ade52554fe89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.157377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f5cf8-7d18-48fe-8e71-ade52554fe89 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.172862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1179/2334] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:43.175911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.187170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ae2e-0fc2-47ad-8d82-591164381954 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.193156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.193868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.194382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.194465 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] PATH_INFO: `/v3/projects/2255a7f7bfd74384b2bf39534a714c19/users/92f527cf06c24bff97452a5f6c98e158/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.212781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:43.221592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2255a7f7bfd74384b2bf39534a714c19, user_id=92f527cf06c24bff97452a5f6c98e158, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.224694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.253833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1156/2335] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:43 2022] PUT /identity/v3/projects/2255a7f7bfd74384b2bf39534a714c19/users/92f527cf06c24bff97452a5f6c98e158/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:43.256691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99c298-bc7b-4d6e-85f1-5f06019a9113 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.257288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99c298-bc7b-4d6e-85f1-5f06019a9113 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.257577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99c298-bc7b-4d6e-85f1-5f06019a9113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.299373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9f99c298-bc7b-4d6e-85f1-5f06019a9113 None None] MFA Rules not processed for user `92f527cf06c24bff97452a5f6c98e158`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:43.302287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9f99c298-bc7b-4d6e-85f1-5f06019a9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:43.412451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1180/2336] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:32:43.424734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.427134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:43.513836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43eac1-6ce0-4d0f-9b50-d0c3e02c90b8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.521973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.522832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.523136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.523638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.536222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.538839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.554676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1157/2337] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:47.098787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71397089-119f-4b69-961b-71611719036c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.099321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71397089-119f-4b69-961b-71611719036c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.099321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71397089-119f-4b69-961b-71611719036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.139658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-71397089-119f-4b69-961b-71611719036c None None] MFA Rules not processed for user `37b485ad806647f595c5b359505f1dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:47.141067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71397089-119f-4b69-961b-71611719036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:47.202182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:47.227707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1181/2338] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:47.229359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:47.231380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:47.315368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e47a-7708-4557-96a1-6f6f48543172 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:47.322535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:47.323662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.324337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.324337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.325119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:47.327076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:47.371173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1158/2339] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:32:47 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:47.375577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71397089-119f-4b69-961b-71611719036c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:47.388086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71397089-119f-4b69-961b-71611719036c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:47.464949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:47.465570 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.465851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.466125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.466766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:47.469524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:47.498064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1182/2340] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:32:47 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:32:47.502451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:47.516385 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a387ae-dec2-42fa-9b4e-502b376720c3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:47.522370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:47.523009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.523292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.523566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.524387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:47.526084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:47.542520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1159/2341] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:47.544152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:47.556167 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75534-c46a-4b23-a7be-4e24e34322e6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:47.561580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:47.562902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.563297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.564021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] PATH_INFO: `/v3/projects/19a5d267c5ef4a12a1d09e51a195497a/users/9fd59ba7f4c24366bca837c3db8f99e0/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.582881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a5d267c5ef4a12a1d09e51a195497a, user_id=9fd59ba7f4c24366bca837c3db8f99e0, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:47.585673 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:47.601910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1183/2342] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:47 2022] PUT /identity/v3/projects/19a5d267c5ef4a12a1d09e51a195497a/users/9fd59ba7f4c24366bca837c3db8f99e0/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:47.606019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:47.618562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfd4b0-44ab-447a-862b-1470ec901fec admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:47.735255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6b9b2-3583-4308-9427-1d2c7fe3c34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:47.735952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6b9b2-3583-4308-9427-1d2c7fe3c34c admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.736253 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6b9b2-3583-4308-9427-1d2c7fe3c34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.736603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6b9b2-3583-4308-9427-1d2c7fe3c34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.737364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6b9b2-3583-4308-9427-1d2c7fe3c34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:47.739057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6b9b2-3583-4308-9427-1d2c7fe3c34c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:47.753170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1160/2343] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:47.757929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:47.769493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4994baa-8f16-4be2-a610-23364a921d28 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:47.774615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:47.775385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:47.775780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:47.776130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] PATH_INFO: `/v3/projects/19a5d267c5ef4a12a1d09e51a195497a/users/9fd59ba7f4c24366bca837c3db8f99e0/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:47.778333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a5d267c5ef4a12a1d09e51a195497a, user_id=9fd59ba7f4c24366bca837c3db8f99e0, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:47.781278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:47.799353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1184/2344] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:47 2022] PUT /identity/v3/projects/19a5d267c5ef4a12a1d09e51a195497a/users/9fd59ba7f4c24366bca837c3db8f99e0/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:32:53.057953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556be3-739a-4c0f-bca1-76f865660917 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:53.058433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556be3-739a-4c0f-bca1-76f865660917 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:53.058617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556be3-739a-4c0f-bca1-76f865660917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:53.067678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:53.095463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-0b556be3-739a-4c0f-bca1-76f865660917 None None] MFA Rules not processed for user `9fd59ba7f4c24366bca837c3db8f99e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:53.096730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0b556be3-739a-4c0f-bca1-76f865660917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:53.240416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1161/2345] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:32:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:53.254745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:53.257414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:53.341855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c35f6-b8b1-4b2d-bff0-8f2dca89aac6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:53.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:53.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:53.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:53.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:53.441799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:53.443225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:53.447251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:53.460097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1185/2346] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:53.653525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a875d7-e720-44f3-adcd-0441fdfc5ea3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:53.653881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a875d7-e720-44f3-adcd-0441fdfc5ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:53.654844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a875d7-e720-44f3-adcd-0441fdfc5ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:53.685515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e1a875d7-e720-44f3-adcd-0441fdfc5ea3 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:53.685736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e1a875d7-e720-44f3-adcd-0441fdfc5ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:53.760962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1162/2347] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:32:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:53.770122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:53.771812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:53.853098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249488e-44ab-4c01-9f2e-66bee5493b1f service glance] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:53.858777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032081b2-56d2-4241-b168-01c00bea793c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:53.859399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032081b2-56d2-4241-b168-01c00bea793c service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:53.859678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032081b2-56d2-4241-b168-01c00bea793c service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:53.859948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032081b2-56d2-4241-b168-01c00bea793c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:53.873464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032081b2-56d2-4241-b168-01c00bea793c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:53.875109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032081b2-56d2-4241-b168-01c00bea793c service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:53.883950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1186/2348] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:33:03.179362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:03.179934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:03.179934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:03.200184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] MFA Rules not processed for user `37b485ad806647f595c5b359505f1dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:03.201463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:03.208275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1163/2349] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:33:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:33:03.880252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=33 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:05.569619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:05.569619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:05.569619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:05.590029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:05.591352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:05.599170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1187/2350] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:33:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:33:05.612852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:05.626213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46171e8-5edb-434d-b481-1c363cde209a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:05.631377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be017e-3307-426b-aad7-2fc6ee0a9bf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:05.632087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be017e-3307-426b-aad7-2fc6ee0a9bf2 service swift] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:05.632442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be017e-3307-426b-aad7-2fc6ee0a9bf2 service swift] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:05.632802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be017e-3307-426b-aad7-2fc6ee0a9bf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:05.641425 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be017e-3307-426b-aad7-2fc6ee0a9bf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:05.643352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be017e-3307-426b-aad7-2fc6ee0a9bf2 service swift] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:05.654303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1164/2351] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:33:15.584110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:15.650478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=66 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:22.470193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:22.556019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d035429-48f5-4d4f-a35d-a71ab685ff8f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:22.562641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:22.563279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:22.563550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:22.563818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] PATH_INFO: `/v3/users/92f527cf06c24bff97452a5f6c98e158` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:22.565015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92f527cf06c24bff97452a5f6c98e158)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:22.567228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:22.623690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92f527cf06c24bff97452a5f6c98e158'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:22.636745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92f527cf06c24bff97452a5f6c98e158'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:22.652799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92f527cf06c24bff97452a5f6c98e158'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:22.653860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92f527cf06c24bff97452a5f6c98e158'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:22.661225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92f527cf06c24bff97452a5f6c98e158'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:22.668306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1188/2352] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:33:22 2022] DELETE /identity/v3/users/92f527cf06c24bff97452a5f6c98e158 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:33:22.674005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:22.674529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:22.674644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:22.699143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:22.700359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:22.769730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1165/2353] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:33:22.777637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:22.779391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:22.846155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91b3e3-f6c8-479d-a4bb-a4600f2a91c6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:22.849787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b260862-7cb7-4266-b925-1d42ec2c0b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:22.851141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b260862-7cb7-4266-b925-1d42ec2c0b0e service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:22.851458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b260862-7cb7-4266-b925-1d42ec2c0b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:22.851738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b260862-7cb7-4266-b925-1d42ec2c0b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:22.863697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b260862-7cb7-4266-b925-1d42ec2c0b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:22.865171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b260862-7cb7-4266-b925-1d42ec2c0b0e service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:22.878428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1189/2354] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:33:23.227212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:23.229249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:23.243322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc47e24-5a96-47ac-8da5-2955db18602a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:23.248290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:23.249023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:23.249362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:23.249720 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] PATH_INFO: `/v3/projects/2255a7f7bfd74384b2bf39534a714c19` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:23.250921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2255a7f7bfd74384b2bf39534a714c19)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:23.252875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:23.313233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2255a7f7bfd74384b2bf39534a714c19', 'request_id': 'req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:23.320947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2255a7f7bfd74384b2bf39534a714c19', 'request_id': 'req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:23.330142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2255a7f7bfd74384b2bf39534a714c19', 'request_id': 'req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:23.346714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] The token cache is being invalidate because project 2255a7f7bfd74384b2bf39534a714c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:33:23.347627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:23.348021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:33:23.351015 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1166/2355] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:33:23 2022] DELETE /identity/v3/projects/2255a7f7bfd74384b2bf39534a714c19 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:33:24.243349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:24.243349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:24.243803 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:24.263088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:24.264366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:24.282085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:24.328491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1190/2356] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:33:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:33:24.352634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:24.354543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:24.426605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bb8aa-488d-4cb9-a433-c040dd16f67d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:24.431812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:24.432418 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:24.432693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:24.432993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:24.441070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:24.442646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:24.451120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1167/2357] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:33:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:33:34.321709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=15 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:34.447966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:36.383754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:36.385268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:36.459478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c4f4-4bc0-41e3-aa77-e176edf49c39 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:36.465320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:36.465926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:36.466229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:36.466499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:36.468197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:36.481385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:36.482967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2be8-5b41-4a5c-a113-c1c822383bfb service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:36.493374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1191/2358] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:33:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:33:38.924700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:38.926251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:38.990579 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47537-66dd-4095-b9b6-c722687c6dab service swift] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:38.993728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ee5f3-005a-4efe-8af6-35b11ee1dcfa None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:38.994316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:38.994701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ee5f3-005a-4efe-8af6-35b11ee1dcfa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:38.994924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:38.995231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:38.995507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:38.995735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ee5f3-005a-4efe-8af6-35b11ee1dcfa None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:38.997290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:38.997662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1192/2359] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:33:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:33:39.003585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0ec80-62b8-437c-a0fd-7f3cf314e867 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.004674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0ec80-62b8-437c-a0fd-7f3cf314e867 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.005123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0ec80-62b8-437c-a0fd-7f3cf314e867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.027175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-96c0ec80-62b8-437c-a0fd-7f3cf314e867 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:39.028666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-96c0ec80-62b8-437c-a0fd-7f3cf314e867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:39.069837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:39.073196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:39.084359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1168/2360] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:33:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:33:39.104564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1193/2361] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:33:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:33:39.116399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:39.117803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:39.181982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7709-9a9a-4461-8612-75bf1aaa0093 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:39.185821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b6925-4932-47eb-a296-3e7a7dbd8bf5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:39.186494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6925-4932-47eb-a296-3e7a7dbd8bf5 service glance] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.186838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6925-4932-47eb-a296-3e7a7dbd8bf5 service glance] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.187159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6925-4932-47eb-a296-3e7a7dbd8bf5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.198434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6925-4932-47eb-a296-3e7a7dbd8bf5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:39.200172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6925-4932-47eb-a296-3e7a7dbd8bf5 service glance] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:39.211504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1169/2362] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:33:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:33:39.284691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37acfa-a963-4bf8-a98c-80766a459f10 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.285009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37acfa-a963-4bf8-a98c-80766a459f10 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.285293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37acfa-a963-4bf8-a98c-80766a459f10 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.287562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1194/2363] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:33:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:33:39.292889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54d862-b1c8-415a-8fe2-73684ba2e4c3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.293238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54d862-b1c8-415a-8fe2-73684ba2e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.293520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54d862-b1c8-415a-8fe2-73684ba2e4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.315756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3e54d862-b1c8-415a-8fe2-73684ba2e4c3 None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:39.317746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3e54d862-b1c8-415a-8fe2-73684ba2e4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:39.324573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1170/2364] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:33:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:33:39.337156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37acfa-a963-4bf8-a98c-80766a459f10 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:39.352474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37acfa-a963-4bf8-a98c-80766a459f10 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:39.356280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a216d6-1d01-402d-9c84-b11a80ef53cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:39.356901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a216d6-1d01-402d-9c84-b11a80ef53cd service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.357180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a216d6-1d01-402d-9c84-b11a80ef53cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.357452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a216d6-1d01-402d-9c84-b11a80ef53cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.365502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a216d6-1d01-402d-9c84-b11a80ef53cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:39.366948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a216d6-1d01-402d-9c84-b11a80ef53cd service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:39.375039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1195/2365] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:33:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:33:39.462868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7fae-2771-4432-a1fc-5e0604595940 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.463234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7fae-2771-4432-a1fc-5e0604595940 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.463602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7fae-2771-4432-a1fc-5e0604595940 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.465163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1171/2366] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:33:39.470655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ab5f-d01c-49c4-84c0-064ad1694c54 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.470943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ab5f-d01c-49c4-84c0-064ad1694c54 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.471243 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ab5f-d01c-49c4-84c0-064ad1694c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.492284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3e13ab5f-d01c-49c4-84c0-064ad1694c54 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:39.493520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3e13ab5f-d01c-49c4-84c0-064ad1694c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:39.504220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1196/2367] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:33:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:33:39.515157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc7fae-2771-4432-a1fc-5e0604595940 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:39.527414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc7fae-2771-4432-a1fc-5e0604595940 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:39.532600 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:39.533316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:39.533609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:39.533885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:39.546424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:39.547932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:39.559878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1172/2368] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:33:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:33:47.105795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9d1bb-0f5b-427c-979f-f05194cc6aea None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:47.106254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9d1bb-0f5b-427c-979f-f05194cc6aea None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:47.106564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9d1bb-0f5b-427c-979f-f05194cc6aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:47.126066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-bac9d1bb-0f5b-427c-979f-f05194cc6aea None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:33:47.127476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bac9d1bb-0f5b-427c-979f-f05194cc6aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:33:47.134743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1197/2369] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:33:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:33:47.150045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:33:47.162777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b473fe-519c-4d56-ad99-a4c3bc534ee5 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:33:47.167581 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390705-bfd6-40c3-9ff9-9e494fed54f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:33:47.168197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390705-bfd6-40c3-9ff9-9e494fed54f0 service swift] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:33:47.168470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390705-bfd6-40c3-9ff9-9e494fed54f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:33:47.168737 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390705-bfd6-40c3-9ff9-9e494fed54f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:33:47.177077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e390705-bfd6-40c3-9ff9-9e494fed54f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:33:47.178642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e390705-bfd6-40c3-9ff9-9e494fed54f0 service swift] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:33:47.186475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1173/2370] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:33:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:33:57.121386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=75 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:33:57.183286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:07.749432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:07.750366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:07.751411 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:07.777053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] MFA Rules not processed for user `33b932f3feb342d889155c3c34407a88`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:07.778781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:07.846780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1198/2371] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:34:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:13.699201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45996a6-47b5-435c-a362-b748b8a16b9f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:13.699673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45996a6-47b5-435c-a362-b748b8a16b9f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:13.699760 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45996a6-47b5-435c-a362-b748b8a16b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:13.723629 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-e45996a6-47b5-435c-a362-b748b8a16b9f None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:13.725397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e45996a6-47b5-435c-a362-b748b8a16b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:13.740390 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1174/2372] 10.222.0.81 () {58 vars in 967 bytes} [Wed Aug 24 03:34:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:13.748794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:13.771965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71601d01-9621-4dee-9b28-98afbf3c3be4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:13.782138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9eb3-e9e0-439f-9860-221ef9edee71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:13.782889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9eb3-e9e0-439f-9860-221ef9edee71 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:13.785299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9eb3-e9e0-439f-9860-221ef9edee71 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:13.786129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9eb3-e9e0-439f-9860-221ef9edee71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:13.795309 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd9eb3-e9e0-439f-9860-221ef9edee71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:13.799657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd9eb3-e9e0-439f-9860-221ef9edee71 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:13.809949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1199/2373] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:34:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:34:23.717502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:23.807070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=31 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:27.318652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:27.319203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:27.319203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:27.338647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] MFA Rules not processed for user `66900b813b934514a7920ba975a05ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:27.339863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:27.396356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1175/2374] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:34:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:37.389166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=25 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:05.577560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:05.578448 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:05.578730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:05.598148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] MFA Rules not processed for user `1af20de2433c44ff8fb1f16290404c21`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:05.599366 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:05.662772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1200/2375] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:10.611352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:10.621542 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07504e1c-c69a-4c8d-a7f6-f44c43e8a202 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:10.625101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:10.625690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:10.625970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:10.626242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:10.627983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:10.641819 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:10.643246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:10.654224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1176/2376] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:11.374208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:11.383662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab0198-87fb-4ca3-ab1a-5521b44a1369 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:11.387912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:11.388506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:11.388812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:11.389198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:11.390820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:11.405410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:11.406880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872ea3-0a9c-4622-bf93-129dee16d626 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:11.418949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1201/2377] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:12.597786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:12.599204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:12.615601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea932f-514e-42d4-ba4a-37c1df99e203 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:12.622386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:12.622970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:12.623247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:12.623516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:12.625318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:12.635973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:12.637606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:12.646782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1177/2378] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:17.759190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:17.759667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:17.759750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:17.784086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:17.785341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:17.793585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1202/2379] 10.222.0.81 () {58 vars in 967 bytes} [Wed Aug 24 03:35:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:17.847125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:17.858308 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd269f-52b4-498b-acc9-5244d0b7a1e9 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:17.863833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:17.864428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:17.864707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:17.865021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:17.874372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:17.876360 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:17.887675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1178/2380] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:35:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:35:18.630755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:18.640335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703839-16ba-4333-98ab-9a6eec32b1de None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:18.644033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:18.644694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:18.645073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:18.645439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:18.647237 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:18.713009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:18.714467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:18.724894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1203/2381] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:22.849641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:22.851181 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:22.867415 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeb25b-bbb8-426d-82c2-07b8f287afbc service swift] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:22.873601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:22.874240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:22.874518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:22.874789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] PATH_INFO: `/v3/users/33b932f3feb342d889155c3c34407a88` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:22.875943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b932f3feb342d889155c3c34407a88)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:22.877931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:22.938575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b932f3feb342d889155c3c34407a88'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:22.945812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b932f3feb342d889155c3c34407a88'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:22.946524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b932f3feb342d889155c3c34407a88'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:22.954602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b932f3feb342d889155c3c34407a88'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:22.973847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-c833466f-8570-4e72-91cf-b0cb55148f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b932f3feb342d889155c3c34407a88'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:22.981765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1179/2382] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:35:22 2022] DELETE /identity/v3/users/33b932f3feb342d889155c3c34407a88 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:23.166232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.185859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196793f-b454-462c-9ac6-5596fabdd4c1 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.261054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.261860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.262136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.262410 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] PATH_INFO: `/v3/projects/625e6391470e4018a6c502219b3d2b15` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.263541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=625e6391470e4018a6c502219b3d2b15)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.265525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.290317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:23.315546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625e6391470e4018a6c502219b3d2b15', 'request_id': 'req-25dd272c-3735-4660-901f-1f539278f55d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:23.321866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625e6391470e4018a6c502219b3d2b15', 'request_id': 'req-25dd272c-3735-4660-901f-1f539278f55d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:23.329870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625e6391470e4018a6c502219b3d2b15', 'request_id': 'req-25dd272c-3735-4660-901f-1f539278f55d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:23.341836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] The token cache is being invalidate because project 625e6391470e4018a6c502219b3d2b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:35:23.342040 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:23.342710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:23.344598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1204/2383] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:35:23 2022] DELETE /identity/v3/projects/625e6391470e4018a6c502219b3d2b15 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:23.349462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdae6ef-d4c9-4a86-b735-11036c7fdddc None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.349754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdae6ef-d4c9-4a86-b735-11036c7fdddc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.350033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdae6ef-d4c9-4a86-b735-11036c7fdddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.376007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-abdae6ef-d4c9-4a86-b735-11036c7fdddc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:23.378506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-abdae6ef-d4c9-4a86-b735-11036c7fdddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:23.379191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.380824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:23.454163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1180/2384] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:23.461812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.463667 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.465549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.494600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd272c-3735-4660-901f-1f539278f55d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.504615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.506358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.506705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.506910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.507901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:23.510113 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:23.511044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:23.527867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1181/2385] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:35:23.529268 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.540789 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc9665-9eb8-4827-9d20-b0a71dd21bf3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.551652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.552748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.553123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.553505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.554302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:35:23.554691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.556491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.571399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1182/2386] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:35:23.576404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.589333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f5555-e157-41c5-bda0-952859eba665 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.595796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.596470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.596884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.597262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.598002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.601426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.602877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.605093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.618935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1205/2387] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:23.621744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:23.646105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1183/2388] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:35:23 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:23.647517 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.667851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf50205-967b-494d-b9fe-1000456a3dfd service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.735265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:23.756011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.756584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.757026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.757306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.757937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.759946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.784925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1206/2389] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:35:23 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:23.787547 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.798895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52347922-97cc-4429-b30e-04b730b249ff admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.803785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.804495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.804883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.805228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.805951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.807646 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.819148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1184/2390] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:35:23.823578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.846938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb22a-94c9-4046-8d5d-3e2598ce6b80 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.852110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79699eef-1e7b-48f3-b0dd-39b42e54a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.852850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79699eef-1e7b-48f3-b0dd-39b42e54a26e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.853149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79699eef-1e7b-48f3-b0dd-39b42e54a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.853421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79699eef-1e7b-48f3-b0dd-39b42e54a26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.854069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79699eef-1e7b-48f3-b0dd-39b42e54a26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.856293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79699eef-1e7b-48f3-b0dd-39b42e54a26e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:23.870241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1207/2391] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:35:23.875460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:23.913398 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef450e-d5d4-412f-9221-08c89d0dde67 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:23.926496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:23.928700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:23.929190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:23.929825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] PATH_INFO: `/v3/projects/b58ccce3d536461891e873ae4f6231d1/users/52b58d18da944243b61434469dcb86c1/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:23.969040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58ccce3d536461891e873ae4f6231d1, user_id=52b58d18da944243b61434469dcb86c1, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:23.972031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.010455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1185/2392] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:35:23 2022] PUT /identity/v3/projects/b58ccce3d536461891e873ae4f6231d1/users/52b58d18da944243b61434469dcb86c1/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:35:24.012539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.013124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.013557 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.046285 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] MFA Rules not processed for user `52b58d18da944243b61434469dcb86c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:24.047656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:24.151768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1208/2393] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:24.166140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.251666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a00c-e53d-4edf-9c10-11c2c3c5045b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:24.257824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:24.258635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.259062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.259478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.260250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:24.262755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.308683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.309293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1186/2394] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:35:24 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:24.320798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b0457-e57f-45fe-a662-eed41a13eff1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:24.402086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:24.403476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.403757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.404029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.404659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:24.406677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.446611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1209/2395] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:35:24 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:35:24.447958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.474434 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76d962-bb59-4a60-8fe4-fcb89b646f00 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:24.489468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:24.490043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.490319 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.490599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.491258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:24.492918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.510228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1187/2396] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:35:24.511344 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.523144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2b2e-a07b-4b9b-a41d-6be886eafffb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:24.529545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:24.530602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.531069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.531532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] PATH_INFO: `/v3/projects/9a320169405b481abd007a0954511364/users/548c2fd1f209463d92a8b53719686e0f/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.550515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a320169405b481abd007a0954511364, user_id=548c2fd1f209463d92a8b53719686e0f, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:24.554453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.572238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1210/2397] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:35:24 2022] PUT /identity/v3/projects/9a320169405b481abd007a0954511364/users/548c2fd1f209463d92a8b53719686e0f/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:35:24.576275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.592852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8051-15ff-477e-843e-0fef454043bc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:24.677927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abca8ea-846f-48e2-9675-fb702097606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:24.679590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca8ea-846f-48e2-9675-fb702097606d admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.680652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca8ea-846f-48e2-9675-fb702097606d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.681231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca8ea-846f-48e2-9675-fb702097606d admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.682352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca8ea-846f-48e2-9675-fb702097606d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:24.687923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca8ea-846f-48e2-9675-fb702097606d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.708830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1188/2398] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:35:24.710852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.725405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e4c24-3c7e-434e-be86-7aa8a1d1fac9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:24.729991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:24.731178 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.731476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.731824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] PATH_INFO: `/v3/projects/9a320169405b481abd007a0954511364/users/548c2fd1f209463d92a8b53719686e0f/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.733862 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a320169405b481abd007a0954511364, user_id=548c2fd1f209463d92a8b53719686e0f, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:24.736653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:24.777640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1211/2399] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:35:24 2022] PUT /identity/v3/projects/9a320169405b481abd007a0954511364/users/548c2fd1f209463d92a8b53719686e0f/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:24.779808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:24.780331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:24.780869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:24.828784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] MFA Rules not processed for user `548c2fd1f209463d92a8b53719686e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:24.830123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:24.846300 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:24.917497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1189/2400] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:24.931796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:24.935070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:24.948433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:25.021683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406e16e-1d8a-4eb7-9a11-16ff012dc6d9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:25.029626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:25.031522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:25.031974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:25.033195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:25.045259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:25.048543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:25.070361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1212/2401] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:25.177975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:25.179433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:25.198433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110da1fc-0340-496e-8b91-53cfe48ea55e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:25.204892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:25.205525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:25.205804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:25.206080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:25.307510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:25.309949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:25.322954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1190/2402] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:26.188696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:26.190288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:26.329471 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9dd40-45ac-4823-b31e-04505d4bdfb7 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:26.337289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:26.338121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:26.338494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:26.338857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] PATH_INFO: `/v3/users/37b485ad806647f595c5b359505f1dd0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:26.340697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37b485ad806647f595c5b359505f1dd0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:26.342810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:26.360779 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:26.362428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:26.404509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37b485ad806647f595c5b359505f1dd0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.413766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37b485ad806647f595c5b359505f1dd0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.430261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37b485ad806647f595c5b359505f1dd0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.431153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37b485ad806647f595c5b359505f1dd0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.440678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37b485ad806647f595c5b359505f1dd0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.446846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1213/2403] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:35:26 2022] DELETE /identity/v3/users/37b485ad806647f595c5b359505f1dd0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:26.448273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e7b57-a205-455d-ac4d-31b2306a488f service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:26.513271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:26.513894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:26.514173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:26.514448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:26.516288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:26.528373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:26.529929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:26.539573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1191/2404] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:35:26.664983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:26.673492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:26.721799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:26.769004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dae33-8f15-4f62-861f-ae29ace4bb40 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:26.771369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:26.772116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:26.772498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:26.772929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] PATH_INFO: `/v3/projects/60ed59a87fd44afb9064c8eef04b19c1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:26.774477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ed59a87fd44afb9064c8eef04b19c1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:26.777524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:26.854110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ed59a87fd44afb9064c8eef04b19c1', 'request_id': 'req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.862937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ed59a87fd44afb9064c8eef04b19c1', 'request_id': 'req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.877151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ed59a87fd44afb9064c8eef04b19c1', 'request_id': 'req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.900257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] The token cache is being invalidate because project 60ed59a87fd44afb9064c8eef04b19c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:35:26.900660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.901331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:26.907041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1214/2405] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:35:26 2022] DELETE /identity/v3/projects/60ed59a87fd44afb9064c8eef04b19c1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:26.908322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:26.921859 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa2c08-2f09-4b6f-812e-2d5bd2a6d333 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:26.927539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:26.990662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:26.991307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:26.991766 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:26.992275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] PATH_INFO: `/v3/users/9fd59ba7f4c24366bca837c3db8f99e0` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:26.993751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd59ba7f4c24366bca837c3db8f99e0)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:26.996362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.067775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd59ba7f4c24366bca837c3db8f99e0'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.078945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd59ba7f4c24366bca837c3db8f99e0'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.081293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd59ba7f4c24366bca837c3db8f99e0'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.096962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd59ba7f4c24366bca837c3db8f99e0'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.112297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-655505f1-f56c-4eb5-a669-690c46b9aa88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd59ba7f4c24366bca837c3db8f99e0'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.123877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1192/2406] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:35:26 2022] DELETE /identity/v3/users/9fd59ba7f4c24366bca837c3db8f99e0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:27.316872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.318648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:27.333461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d007c-6e94-48da-a23d-ceb6cb5da6fe admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.393992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.394617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.394894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.395161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] PATH_INFO: `/v3/projects/19a5d267c5ef4a12a1d09e51a195497a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.396265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a5d267c5ef4a12a1d09e51a195497a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.398215 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.447853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a5d267c5ef4a12a1d09e51a195497a', 'request_id': 'req-f5326559-80f1-4326-a7d6-00f0cb3dcc40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.455086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a5d267c5ef4a12a1d09e51a195497a', 'request_id': 'req-f5326559-80f1-4326-a7d6-00f0cb3dcc40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.464263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a5d267c5ef4a12a1d09e51a195497a', 'request_id': 'req-f5326559-80f1-4326-a7d6-00f0cb3dcc40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.466895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:27.481655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] The token cache is being invalidate because project 19a5d267c5ef4a12a1d09e51a195497a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:35:27.482039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.482787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f5326559-80f1-4326-a7d6-00f0cb3dcc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:35:27.484541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1215/2407] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:35:27 2022] DELETE /identity/v3/projects/19a5d267c5ef4a12a1d09e51a195497a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:27.489997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.490472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.490565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.514497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:27.516131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:27.576721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1193/2408] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:27.585475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.585995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.586324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.608267 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:27.609537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:27.617527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1216/2409] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:27.622331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.632006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061cb2d-f7f3-471f-9bf5-0fd75aed362a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.637120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.637854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.638140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.638414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.639061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:35:27.639364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.640989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.653369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1194/2410] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:35:27.657756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.667063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7da7f-0803-4ed6-8469-b6694558b430 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.672254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.672870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.673146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.673416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.674039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.675953 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.703841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1217/2411] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:35:27 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:27.708157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.719410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01db70-7a65-47de-b6b6-950dd88b5a78 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.731528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:27.780195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.780781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.781168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.781384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.782063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.784027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.806005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1195/2412] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:35:27 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:27.811564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.828721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dea3c-8aa9-4fd0-a21b-366e94810daf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.839808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.840426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.840702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.841010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.841652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.843502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.853898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1218/2413] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:35:27.858328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.868176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44bbaf-f93e-443a-b703-646671f88ae4 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.873596 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ffbe8-d496-4ee8-b377-438acb73bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.874173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ffbe8-d496-4ee8-b377-438acb73bd31 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.874447 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ffbe8-d496-4ee8-b377-438acb73bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.874712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ffbe8-d496-4ee8-b377-438acb73bd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.875430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ffbe8-d496-4ee8-b377-438acb73bd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.877328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ffbe8-d496-4ee8-b377-438acb73bd31 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.888470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1196/2414] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:35:27.897613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:27.908734 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbfad0-61ce-4f23-90a1-72b2721ab187 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:27.913580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:27.914226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.914503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.914778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] PATH_INFO: `/v3/projects/5dea7fb32dea4f75917d82973bf7e7e6/users/e5588121cc364d30a8f01d17c4cfc6df/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.930145 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dea7fb32dea4f75917d82973bf7e7e6, user_id=e5588121cc364d30a8f01d17c4cfc6df, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:27.932877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:27.950529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1219/2415] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:35:27 2022] PUT /identity/v3/projects/5dea7fb32dea4f75917d82973bf7e7e6/users/e5588121cc364d30a8f01d17c4cfc6df/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:27.955637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54427-a354-4641-8272-c0e002f3b386 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:27.955991 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54427-a354-4641-8272-c0e002f3b386 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:27.956269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54427-a354-4641-8272-c0e002f3b386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:27.982488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d1d54427-a354-4641-8272-c0e002f3b386 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:27.983604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d1d54427-a354-4641-8272-c0e002f3b386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:28.046073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1197/2416] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:28.055594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:28.057294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:28.118123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1979f-8caf-4514-8175-c497aee55c1f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:28.122825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:28.123430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:28.123707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:28.123979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:28.133005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:28.134431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:28.144235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1220/2417] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:32.685652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:32.686102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:32.686197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:32.719858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] MFA Rules not processed for user `e5588121cc364d30a8f01d17c4cfc6df`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:32.721569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:32.818031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1198/2418] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:32.823056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:32.833957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161212d-585b-45c1-a3d7-fbc75df5d2de service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:32.838765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:32.839595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:32.839978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:32.840372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:32.841180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:32.843457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:32.874131 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1221/2419] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:35:32 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:32.878462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:32.893810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ee81-bf63-4a51-a480-5e1d75f73938 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:32.915518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:32.957821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:32.958337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:32.958618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:32.958924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:32.959562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:32.961575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:32.988507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1199/2420] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:35:32 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:35:32.989731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:33.001162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea3fe-3e9c-426b-a273-2e3986e96174 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:33.005925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:33.006548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:33.006833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:33.007153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:33.007831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:33.009520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:33.021206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1222/2421] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:35:33.025281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:33.036440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02527f1e-a351-4128-8bcb-8e2792dadc42 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:33.041361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:33.042127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:33.042941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:33.043221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] PATH_INFO: `/v3/projects/9d0d9fcf69f04ad99a8c1e68eea71735/users/eba4b2752d54437ea93823e52f76b32b/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:33.059804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0d9fcf69f04ad99a8c1e68eea71735, user_id=eba4b2752d54437ea93823e52f76b32b, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:33.062724 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:33.078109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1200/2422] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:35:33 2022] PUT /identity/v3/projects/9d0d9fcf69f04ad99a8c1e68eea71735/users/eba4b2752d54437ea93823e52f76b32b/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:33.082228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:33.094229 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb58167-ea4b-495b-ac0c-3833b118488f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:33.149043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:33.158899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8b0e-44ad-4aec-bd1a-2206baa20b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:33.160096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8b0e-44ad-4aec-bd1a-2206baa20b44 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:33.160463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8b0e-44ad-4aec-bd1a-2206baa20b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:33.161034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8b0e-44ad-4aec-bd1a-2206baa20b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:33.161682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8b0e-44ad-4aec-bd1a-2206baa20b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:33.163377 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8b0e-44ad-4aec-bd1a-2206baa20b44 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:33.176612 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1223/2423] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:35:33.180665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:33.195612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f9b52-1ff9-45a4-a047-092eab9070a3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:33.203806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:33.204645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:33.204959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:33.205289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] PATH_INFO: `/v3/projects/9d0d9fcf69f04ad99a8c1e68eea71735/users/eba4b2752d54437ea93823e52f76b32b/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:33.207365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0d9fcf69f04ad99a8c1e68eea71735, user_id=eba4b2752d54437ea93823e52f76b32b, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:33.210255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:33.223650 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1201/2424] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:35:33 2022] PUT /identity/v3/projects/9d0d9fcf69f04ad99a8c1e68eea71735/users/eba4b2752d54437ea93823e52f76b32b/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:35:38.054173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b41979-f1bf-423e-a996-74bc5fac62da None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:38.054664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b41979-f1bf-423e-a996-74bc5fac62da None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:38.054664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b41979-f1bf-423e-a996-74bc5fac62da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:38.089596 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-52b41979-f1bf-423e-a996-74bc5fac62da None None] MFA Rules not processed for user `eba4b2752d54437ea93823e52f76b32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:38.091404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-52b41979-f1bf-423e-a996-74bc5fac62da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:38.170871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1224/2425] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:38.190959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:38.192963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:38.285745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a763a-b9ec-4cae-8e41-9c12f3705cd8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:38.294844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:38.295620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:38.295903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:38.296179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:38.374436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:38.376182 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:38.394515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1202/2426] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:39.568013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:39.568476 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:39.568584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:39.590773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] MFA Rules not processed for user `e5588121cc364d30a8f01d17c4cfc6df`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:35:39.592104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:39.599796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1225/2427] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:35:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:35:48.388168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:49.584175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=41 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:54.958098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:54.970630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f621d12-9aad-4b17-ad77-297e89479161 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:54.986971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:54.987771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:54.988092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:54.988372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:54.990616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:55.022307 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:55.024137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:55.057652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1203/2428] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:59.694948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:35:59.756159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a127-93a6-415e-a931-bdb6c9c1cccd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:35:59.759617 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:35:59.760213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:35:59.760491 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:35:59.760785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:35:59.763514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:35:59.772723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:35:59.774179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:35:59.783011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1226/2429] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:35:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:00.012717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:00.021995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a71ab1-c29f-4ed0-a575-09838bc5ff0e service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:00.025850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:00.026445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:00.027280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:00.027662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:00.029796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:00.096076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:00.097557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:00.108661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1204/2430] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:00.250914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:00.250914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:00.351407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf509ae-93a5-4658-8ddd-0f54215b8dd8 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:00.357818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:00.358503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:00.358847 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:00.359191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:00.361415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:00.375896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:00.377541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:00.388316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1227/2431] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:03.064428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:03.077895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d74b6-4d4d-48dc-b1f3-1913408220e5 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:03.083890 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:03.084510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:03.085067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:03.085436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:03.087437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:03.101123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:03.102910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:03.115389 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1205/2432] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:03.128097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:03.129956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:03.147025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f69af-0c88-49ba-9cee-416d055f1e75 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:03.152945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:03.153710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:03.154084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:03.154459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] PATH_INFO: `/v3/users/66900b813b934514a7920ba975a05ec5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:03.155699 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66900b813b934514a7920ba975a05ec5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:03.157952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:03.206236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66900b813b934514a7920ba975a05ec5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.213337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66900b813b934514a7920ba975a05ec5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.225938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66900b813b934514a7920ba975a05ec5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.226585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66900b813b934514a7920ba975a05ec5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.233726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-401803af-2c72-48a7-9969-dcd61dc7d38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66900b813b934514a7920ba975a05ec5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.239662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1228/2433] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:03 2022] DELETE /identity/v3/users/66900b813b934514a7920ba975a05ec5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:03.686687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:03.699232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc6569-3ee6-409d-a400-ede18f107015 service placement] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:03.765887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:03.766505 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:03.766781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:03.767062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] PATH_INFO: `/v3/projects/6a1e9994c1c045c892059d5b80578ba7` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:03.768309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1e9994c1c045c892059d5b80578ba7)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:03.770232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:03.830827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1e9994c1c045c892059d5b80578ba7', 'request_id': 'req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.841610 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1e9994c1c045c892059d5b80578ba7', 'request_id': 'req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.853502 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1e9994c1c045c892059d5b80578ba7', 'request_id': 'req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.865077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:03.873381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] The token cache is being invalidate because project 6a1e9994c1c045c892059d5b80578ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:03.873894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.875404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-406d7b4e-87b4-41fe-97e6-aa0b00b8214f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:03.878812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1206/2434] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:03 2022] DELETE /identity/v3/projects/6a1e9994c1c045c892059d5b80578ba7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:03.883719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:03.884062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:03.884384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:03.895940 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:03.914152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:03.915538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:04.003931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1229/2435] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:04.006173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.006557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.006931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.040086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:04.041681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:04.060137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1207/2436] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:04.061121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.090111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e8cf0-8a96-4979-9a55-67679768c501 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.102108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.102720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.102994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.103261 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.104015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:04.104193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.106710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.139331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1230/2437] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:04.144152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.158396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e528187-47b3-4872-93ea-635c7832f0b2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.164887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.166077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.166541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.167204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.168746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.172115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.217306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1208/2438] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:04 2022] POST /identity/v3/projects => generated 405 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:04.218578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.230836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37bc83-860c-4b3c-a4f6-19188f32ae4b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.299628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.300239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.300603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.300979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.301659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.303705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.325035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1231/2439] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:04 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:04.329267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.340975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c791a-eb2b-42cd-8a1b-67b52f1cfc25 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.345938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.346540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.346820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.347092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.347735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.349458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.359738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1209/2440] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:04.363635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.374687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a060c07-4d22-45b6-b0fd-1ac9bd83b49d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.379386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879cd60-1031-4136-a74e-8dc9d9c2cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.379975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879cd60-1031-4136-a74e-8dc9d9c2cd0b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.380275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879cd60-1031-4136-a74e-8dc9d9c2cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.380546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879cd60-1031-4136-a74e-8dc9d9c2cd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.381211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879cd60-1031-4136-a74e-8dc9d9c2cd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.382814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879cd60-1031-4136-a74e-8dc9d9c2cd0b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.394737 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1232/2441] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:04.398804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.409706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d4c85-b1d5-4d10-b7b8-32e7d1d81ac4 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.414686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.415383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.415664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.415944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] PATH_INFO: `/v3/projects/0ff1e3b5db0e4b6e8a3b2524244e5f39/users/390a174f407f4d50871253bd05602197/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.434226 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff1e3b5db0e4b6e8a3b2524244e5f39, user_id=390a174f407f4d50871253bd05602197, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.438014 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.453738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1210/2442] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:04 2022] PUT /identity/v3/projects/0ff1e3b5db0e4b6e8a3b2524244e5f39/users/390a174f407f4d50871253bd05602197/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:04.465873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364fda-4717-4827-a401-50b75d7be0de None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.466156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364fda-4717-4827-a401-50b75d7be0de None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.466434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364fda-4717-4827-a401-50b75d7be0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.495982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-88364fda-4717-4827-a401-50b75d7be0de None None] MFA Rules not processed for user `390a174f407f4d50871253bd05602197`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:04.497260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-88364fda-4717-4827-a401-50b75d7be0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:04.565335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:04.573192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1233/2443] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:04.575031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.646424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e2a8c-e4df-458d-ade4-99f1508403d6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.652364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.653352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.653750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.654140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.654803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.656716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.703699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1211/2444] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:04 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:04.705062 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364fda-4717-4827-a401-50b75d7be0de None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.719927 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364fda-4717-4827-a401-50b75d7be0de None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.829762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.830480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.831628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.832015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.832791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.835709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.869970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1234/2445] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:04 2022] POST /identity/v3/users => generated 309 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:04.874965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.892233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db8828-7095-405f-87fe-e99d4c24d62a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.909897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.909897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.909897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.909897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.910368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.914484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:04.947811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1212/2446] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:04.948996 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:04.960171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3973e-a50b-4018-900f-fff375ff73a5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:04.965593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:04.966259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:04.966537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:04.966812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] PATH_INFO: `/v3/projects/943c00c54eee49dea23093ccc121a9da/users/4d03134edc2a4433850c824b01e74ce6/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:04.983982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=943c00c54eee49dea23093ccc121a9da, user_id=4d03134edc2a4433850c824b01e74ce6, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:04.987567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:05.013750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1235/2447] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:04 2022] PUT /identity/v3/projects/943c00c54eee49dea23093ccc121a9da/users/4d03134edc2a4433850c824b01e74ce6/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:05.020714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:05.040142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8aad3d-b820-41bf-b677-aa69d9916a69 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:05.108371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576755f-c7f9-46c5-b855-3d814b3ea312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:05.109008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576755f-c7f9-46c5-b855-3d814b3ea312 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:05.109283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576755f-c7f9-46c5-b855-3d814b3ea312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:05.109551 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576755f-c7f9-46c5-b855-3d814b3ea312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:05.110190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576755f-c7f9-46c5-b855-3d814b3ea312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:05.111832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576755f-c7f9-46c5-b855-3d814b3ea312 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:05.122626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1213/2448] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:05.126795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:05.138015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc022be-ab9d-4467-a708-859c7eb13533 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:05.142694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:05.143334 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:05.143609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:05.143881 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] PATH_INFO: `/v3/projects/943c00c54eee49dea23093ccc121a9da/users/4d03134edc2a4433850c824b01e74ce6/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:05.145822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] RBAC: Authorizing `identity:create_grant(project_id=943c00c54eee49dea23093ccc121a9da, user_id=4d03134edc2a4433850c824b01e74ce6, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:05.148459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:05.159205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1236/2449] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:05 2022] PUT /identity/v3/projects/943c00c54eee49dea23093ccc121a9da/users/4d03134edc2a4433850c824b01e74ce6/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:05.169942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:05.170449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:05.170632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:05.178861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:05.204393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] MFA Rules not processed for user `4d03134edc2a4433850c824b01e74ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:05.205746 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:05.299251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1214/2450] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:05.311202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:05.312034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:05.388615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240be789-923d-45af-99ab-9f88c360d36d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:05.393595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14a46d-46d9-4107-8f64-2cb0c78937b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:05.394245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14a46d-46d9-4107-8f64-2cb0c78937b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:05.394561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14a46d-46d9-4107-8f64-2cb0c78937b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:05.394806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14a46d-46d9-4107-8f64-2cb0c78937b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:05.405852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14a46d-46d9-4107-8f64-2cb0c78937b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:05.408479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14a46d-46d9-4107-8f64-2cb0c78937b9 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:05.424583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1237/2451] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:05 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:05.708826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:05.783369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e993bc-ab39-4506-83d9-c70bfb5f265f None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:05.787894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:05.788811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:05.789097 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:05.791729 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] PATH_INFO: `/v3/users/390a174f407f4d50871253bd05602197` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:05.793340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] RBAC: Authorizing `identity:delete_user(user_id=390a174f407f4d50871253bd05602197)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:05.795400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:05.847840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390a174f407f4d50871253bd05602197'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:05.855716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390a174f407f4d50871253bd05602197'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:05.856397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390a174f407f4d50871253bd05602197'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:05.865241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390a174f407f4d50871253bd05602197'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:05.878410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390a174f407f4d50871253bd05602197'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:05.888222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1215/2452] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:05 2022] DELETE /identity/v3/users/390a174f407f4d50871253bd05602197 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:05.890361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:05.890950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:05.894571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:05.912667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:05.913757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:05.977277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:05.994287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1238/2453] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:06.004783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:06.006193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:06.017632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:06.093232 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dcf98-5462-4c6e-8a56-7f8dae1ea4be admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:06.097286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:06.098361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:06.098713 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:06.098984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:06.109690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:06.111143 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:06.122451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1216/2454] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:06.450289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:06.451744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:06.463289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbfe77-b609-4c23-9538-4c18a3467d04 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:06.468520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:06.469118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:06.469390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:06.469659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] PATH_INFO: `/v3/projects/0ff1e3b5db0e4b6e8a3b2524244e5f39` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:06.470760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff1e3b5db0e4b6e8a3b2524244e5f39)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:06.472666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:06.530312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff1e3b5db0e4b6e8a3b2524244e5f39', 'request_id': 'req-25d81e40-08ca-47ca-9bb1-f447f352755c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.536552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff1e3b5db0e4b6e8a3b2524244e5f39', 'request_id': 'req-25d81e40-08ca-47ca-9bb1-f447f352755c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.545966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff1e3b5db0e4b6e8a3b2524244e5f39', 'request_id': 'req-25d81e40-08ca-47ca-9bb1-f447f352755c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.556545 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] The token cache is being invalidate because project 0ff1e3b5db0e4b6e8a3b2524244e5f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:06.556859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.557492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.559610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1239/2455] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:06 2022] DELETE /identity/v3/projects/0ff1e3b5db0e4b6e8a3b2524244e5f39 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:06.563714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:06.574190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5c233-f809-475d-8766-6d8d576cc01b service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:06.652636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:06.653340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:06.653652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:06.654019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] PATH_INFO: `/v3/users/4d03134edc2a4433850c824b01e74ce6` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:06.655179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d03134edc2a4433850c824b01e74ce6)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:06.657295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:06.719839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d03134edc2a4433850c824b01e74ce6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.729644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d03134edc2a4433850c824b01e74ce6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.731555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d03134edc2a4433850c824b01e74ce6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.742330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d03134edc2a4433850c824b01e74ce6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.758712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d03134edc2a4433850c824b01e74ce6'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:06.767024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1217/2456] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:06 2022] DELETE /identity/v3/users/4d03134edc2a4433850c824b01e74ce6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:07.029991 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.031443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.046314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d81e40-08ca-47ca-9bb1-f447f352755c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.090404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.092311 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.119042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.119663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.119946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.120957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] PATH_INFO: `/v3/users/52b58d18da944243b61434469dcb86c1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.122347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b58d18da944243b61434469dcb86c1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.125255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.146941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:07.192950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a297d-d6f0-4496-ac55-8bf6b12e8e22 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.198594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b58d18da944243b61434469dcb86c1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.199362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.200201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.200647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.200990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] PATH_INFO: `/v3/projects/943c00c54eee49dea23093ccc121a9da` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.202160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=943c00c54eee49dea23093ccc121a9da)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.204254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.207659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b58d18da944243b61434469dcb86c1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.230784 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b58d18da944243b61434469dcb86c1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.231658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b58d18da944243b61434469dcb86c1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.240439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a93fb077-1351-46cb-96f6-e806baf69ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b58d18da944243b61434469dcb86c1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.242279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:07.251774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1240/2457] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:07 2022] DELETE /identity/v3/users/52b58d18da944243b61434469dcb86c1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:07.256702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.256702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.257183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.273902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943c00c54eee49dea23093ccc121a9da', 'request_id': 'req-d5374899-90b0-464b-bae1-3272886e1f0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.281057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943c00c54eee49dea23093ccc121a9da', 'request_id': 'req-d5374899-90b0-464b-bae1-3272886e1f0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.283100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:07.284509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.290115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943c00c54eee49dea23093ccc121a9da', 'request_id': 'req-d5374899-90b0-464b-bae1-3272886e1f0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.303279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] The token cache is being invalidate because project 943c00c54eee49dea23093ccc121a9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:07.303567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.304267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d5374899-90b0-464b-bae1-3272886e1f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.306149 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1218/2458] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:07 2022] DELETE /identity/v3/projects/943c00c54eee49dea23093ccc121a9da => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:07.311081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839c48-63d6-4075-971e-7c067d100090 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.311363 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839c48-63d6-4075-971e-7c067d100090 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.311638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839c48-63d6-4075-971e-7c067d100090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.336920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a2839c48-63d6-4075-971e-7c067d100090 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:07.338148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a2839c48-63d6-4075-971e-7c067d100090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.364185 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1241/2459] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:07.374397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.375812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.405264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1219/2460] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:07.413112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.413397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.413671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.433782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:07.435390 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.437822 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b132d26-330e-4c31-a2d2-60c484c18088 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.441710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.442310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.442585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.442882 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.445734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1220/2461] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:07.450367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.455911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.457416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.459835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808acae-aaa2-49f3-bf46-12a76aa35d81 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.465460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.466052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.466333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.466605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.467517 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:07.467816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.468779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1242/2462] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:07.470632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.485007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1221/2463] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:36:07.487375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.497591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5c49c-7dd1-4524-8ca7-703720e79be3 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.502462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.503256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.503530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.503800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.504412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.506391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.539697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1243/2464] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:07 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:07.540867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.550970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabfc41-579e-4991-b321-a94f0f1a5c80 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.594687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:07.616559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.616559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.616559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.616559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.617000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.618463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.642450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1222/2465] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:07 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:07.646627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.657361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35c0f8-57ac-48cf-9250-a21a6e72443c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.661986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.662579 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.662852 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.663119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.663759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.665500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.678935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1244/2466] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:07.680285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.694452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4b2f-c0bf-4aac-8061-1e27b6b8001a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.700192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.700979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.701130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.701519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.702034 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.704120 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.717329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1223/2467] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:07.721486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.737175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe35e-e753-4778-b525-09066d8b085c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.742956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07e27b-130e-4671-9079-4ccaf6a1f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.743666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07e27b-130e-4671-9079-4ccaf6a1f911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.743934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07e27b-130e-4671-9079-4ccaf6a1f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.744148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07e27b-130e-4671-9079-4ccaf6a1f911 admin admin] PATH_INFO: `/v3/projects/d994b67a28ac44129851c4d89d71a5e8/users/7668f71a73ad4ed7bc8e3cd08c455032/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.772619 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.774524 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.790677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ff77-46af-4b03-9df3-f1abf764e621 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.795986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:07.796751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.797130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.797477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] PATH_INFO: `/v3/projects/b58ccce3d536461891e873ae4f6231d1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.798775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58ccce3d536461891e873ae4f6231d1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.801627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.862358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58ccce3d536461891e873ae4f6231d1', 'request_id': 'req-43e9c60b-3077-40ce-8e96-f85019fd4613'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.869211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58ccce3d536461891e873ae4f6231d1', 'request_id': 'req-43e9c60b-3077-40ce-8e96-f85019fd4613'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.879090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58ccce3d536461891e873ae4f6231d1', 'request_id': 'req-43e9c60b-3077-40ce-8e96-f85019fd4613'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.890699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] The token cache is being invalidate because project b58ccce3d536461891e873ae4f6231d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:07.890986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.891666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:07.893536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1224/2468] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:07 2022] DELETE /identity/v3/projects/b58ccce3d536461891e873ae4f6231d1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:07.897140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:07.897368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07e27b-130e-4671-9079-4ccaf6a1f911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d994b67a28ac44129851c4d89d71a5e8, user_id=7668f71a73ad4ed7bc8e3cd08c455032, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:07.898461 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.900067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07e27b-130e-4671-9079-4ccaf6a1f911 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:07.915777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1245/2469] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:07 2022] PUT /identity/v3/projects/d994b67a28ac44129851c4d89d71a5e8/users/7668f71a73ad4ed7bc8e3cd08c455032/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:07.927975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:07.928250 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:07.928515 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:07.957290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] MFA Rules not processed for user `7668f71a73ad4ed7bc8e3cd08c455032`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:07.958500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:07.968202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9c60b-3077-40ce-8e96-f85019fd4613 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:07.990944 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:08.017641 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:08.024670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1246/2470] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:08.029755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.029902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.030421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.030705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.030975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] PATH_INFO: `/v3/users/548c2fd1f209463d92a8b53719686e0f` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.031122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.032135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548c2fd1f209463d92a8b53719686e0f)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.034350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.044745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10ee73-c746-4634-b302-17f35ebd25b1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.064782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.065461 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.065771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.066169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.066726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.068754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.097527 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548c2fd1f209463d92a8b53719686e0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.103759 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1247/2471] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:08.105717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548c2fd1f209463d92a8b53719686e0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.106454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548c2fd1f209463d92a8b53719686e0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.109089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.111280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.124220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548c2fd1f209463d92a8b53719686e0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.138692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548c2fd1f209463d92a8b53719686e0f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.145030 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1225/2472] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:07 2022] DELETE /identity/v3/users/548c2fd1f209463d92a8b53719686e0f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:08.186177 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ad0c6-6864-484b-9860-72b0bee5e8a2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.244281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.244916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.245244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.245571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.246187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.248104 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.268494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1248/2473] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:08.272209 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.283785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a73a7-645c-49c1-a13b-83b5c82ff1c8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.288584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.289331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.289684 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.290080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.290732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.293208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.303437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1226/2474] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:08.307884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.319776 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2959-8145-478a-8572-5bd02bf65c81 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.325656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.326333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.326656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.326880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] PATH_INFO: `/v3/projects/1fb395c6bb9644318479228593d1a5b6/users/84496e7152fd41419a37925dfc2f5863/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.343299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb395c6bb9644318479228593d1a5b6, user_id=84496e7152fd41419a37925dfc2f5863, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.346039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.361467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1249/2475] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:08 2022] PUT /identity/v3/projects/1fb395c6bb9644318479228593d1a5b6/users/84496e7152fd41419a37925dfc2f5863/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:08.365393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.376785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204f26-a705-49f1-b160-4be70074427e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.446369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.450231 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.466658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.467660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.468041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.468410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.469347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.471292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.485717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1227/2476] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:08.490118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.491860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:08.505067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142c0f5-4a11-41e9-8d4c-ff013dae7883 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.510864 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bda073-0679-403d-9678-6a25cd864c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.511664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bda073-0679-403d-9678-6a25cd864c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.512042 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bda073-0679-403d-9678-6a25cd864c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.512416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bda073-0679-403d-9678-6a25cd864c97 admin admin] PATH_INFO: `/v3/projects/1fb395c6bb9644318479228593d1a5b6/users/84496e7152fd41419a37925dfc2f5863/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.515050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bda073-0679-403d-9678-6a25cd864c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb395c6bb9644318479228593d1a5b6, user_id=84496e7152fd41419a37925dfc2f5863, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.519047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bda073-0679-403d-9678-6a25cd864c97 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.533426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1228/2477] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:08 2022] PUT /identity/v3/projects/1fb395c6bb9644318479228593d1a5b6/users/84496e7152fd41419a37925dfc2f5863/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:08.538132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a855db-fed5-48bf-b157-8c5ff9b92598 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.545554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.546162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.546692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.578221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] MFA Rules not processed for user `84496e7152fd41419a37925dfc2f5863`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:08.579462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.600717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.601382 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.601661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.601930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] PATH_INFO: `/v3/projects/9a320169405b481abd007a0954511364` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.603063 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a320169405b481abd007a0954511364)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.605499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.613484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:08.659414 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1229/2478] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:08.661414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a320169405b481abd007a0954511364', 'request_id': 'req-cb267797-84a0-4fcb-8d05-db45fd665d09'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.668818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a320169405b481abd007a0954511364', 'request_id': 'req-cb267797-84a0-4fcb-8d05-db45fd665d09'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.670730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.672138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.676924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a320169405b481abd007a0954511364', 'request_id': 'req-cb267797-84a0-4fcb-8d05-db45fd665d09'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.687957 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] The token cache is being invalidate because project 9a320169405b481abd007a0954511364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:08.688255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.688946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cb267797-84a0-4fcb-8d05-db45fd665d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:08.690674 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1250/2479] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:08 2022] DELETE /identity/v3/projects/9a320169405b481abd007a0954511364 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:08.695057 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aba88-8ec8-4f46-93dd-a6fd49f7441d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.695338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aba88-8ec8-4f46-93dd-a6fd49f7441d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.695613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aba88-8ec8-4f46-93dd-a6fd49f7441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.718786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-267aba88-8ec8-4f46-93dd-a6fd49f7441d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:08.719992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-267aba88-8ec8-4f46-93dd-a6fd49f7441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.732966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac5bbe-8402-4cff-86e5-e46241fd36d2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.781980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.782578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.782854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.783123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.783690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1251/2480] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:08.791514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.793800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.794217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.794499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.802629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:08.817691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:08.818887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:08.829738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1252/2481] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:08.833582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.843511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1ef8d-7ffc-4713-b839-323a8254637c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.848294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.848919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.849193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.849462 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.850096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:08.850421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.851316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.852021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.852925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.864480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1253/2482] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:36:08.868130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1230/2483] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:08.869736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.880043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763593-8622-49af-add6-5125695ffcc4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.886793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:08.887489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:08.887841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:08.888516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:08.889343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:08.891386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:08.921311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1254/2484] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:08.925410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:08.936681 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503476dc-460a-47e5-a54f-cd297519e93b service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:08.969939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:09.000839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.001449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.001732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.002008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.002680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.004636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.030660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1231/2485] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:08 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:09.031910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.043390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a29321-16da-44a6-9022-50412f2fae1b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.047942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.048543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.048857 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.049130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.049767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.051406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.062354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1255/2486] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:09.068590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.079993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154860-04aa-4345-b538-2748a55c0807 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.084672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60267d-979b-4641-881b-109a866b4186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.085290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60267d-979b-4641-881b-109a866b4186 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.085568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60267d-979b-4641-881b-109a866b4186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.085833 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60267d-979b-4641-881b-109a866b4186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.086491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60267d-979b-4641-881b-109a866b4186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.090960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60267d-979b-4641-881b-109a866b4186 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.102219 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1232/2487] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:09.106826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.116691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97ef7-5306-4247-b774-44cce824238b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.122406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.123056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.123330 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.123597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] PATH_INFO: `/v3/projects/bffc258aa06a454c83849dc6ff324518/users/08c3e6dcb2e34ce6b54811134ef115c5/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.139782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffc258aa06a454c83849dc6ff324518, user_id=08c3e6dcb2e34ce6b54811134ef115c5, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.142497 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.158380 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1256/2488] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:09 2022] PUT /identity/v3/projects/bffc258aa06a454c83849dc6ff324518/users/08c3e6dcb2e34ce6b54811134ef115c5/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:09.169345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.169635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.169917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.201123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] MFA Rules not processed for user `08c3e6dcb2e34ce6b54811134ef115c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:09.202350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.271105 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1233/2489] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:09.280824 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.280986 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.282350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.351085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9a13b-0408-4036-b9b4-250e4df343e1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.352307 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4101c-e230-498f-b5e3-37a9de275079 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.356292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.356996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.357270 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.357539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] PATH_INFO: `/v3/users/7668f71a73ad4ed7bc8e3cd08c455032` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.357890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.358490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.358666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7668f71a73ad4ed7bc8e3cd08c455032)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.358806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.359074 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.359707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.360604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.361736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.389677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1257/2490] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:09 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:09.393945 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.408691 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:09.411076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7668f71a73ad4ed7bc8e3cd08c455032'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:09.421382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7668f71a73ad4ed7bc8e3cd08c455032'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:09.422099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7668f71a73ad4ed7bc8e3cd08c455032'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:09.430884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7668f71a73ad4ed7bc8e3cd08c455032'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:09.446095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2de977f6-5429-4a83-ba32-2f903bb3125f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7668f71a73ad4ed7bc8e3cd08c455032'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:09.452584 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1234/2491] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:09 2022] DELETE /identity/v3/users/7668f71a73ad4ed7bc8e3cd08c455032 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:09.456862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.457269 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.457576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.464874 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191a1e5-420f-4d4e-b8df-e6741ddc6fc8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.476397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:09.477314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:09.477926 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.524661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.525352 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.525636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.525910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.526785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.529022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.538690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1235/2492] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:09.550221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.551902 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.556008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1258/2493] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:09 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:09.560509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.562140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.573954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad67022-be50-4ee9-a8e7-262cbd8c2698 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.578549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.579133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.579443 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.579716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.580349 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.582195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.595310 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1259/2494] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:09.599221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.612200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6458-c2ef-45cf-bb63-386be13658f0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.617149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.617792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.618506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.618648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9542bcc-3032-4608-b054-b92998da5eea None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.619109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] PATH_INFO: `/v3/projects/ec41de767f324fb8954e2206383a75e3/users/033a39af2f9f42b1b74002fd229fa067/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.625251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.625979 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.626350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.626854 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.636389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec41de767f324fb8954e2206383a75e3, user_id=033a39af2f9f42b1b74002fd229fa067, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.637049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.638494 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.639203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.649801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1236/2495] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:09.659611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1260/2496] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:09 2022] PUT /identity/v3/projects/ec41de767f324fb8954e2206383a75e3/users/033a39af2f9f42b1b74002fd229fa067/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:09.663511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.673772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ec5ec-2363-4aa9-9c1e-30c9030fd08c service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.737127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f549c-84a1-4a49-9b71-ff375191a691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.737425 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f549c-84a1-4a49-9b71-ff375191a691 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.737709 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f549c-84a1-4a49-9b71-ff375191a691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.737983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f549c-84a1-4a49-9b71-ff375191a691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.738631 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f549c-84a1-4a49-9b71-ff375191a691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.740272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f549c-84a1-4a49-9b71-ff375191a691 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.745634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:09.755130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1237/2497] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:09.756157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.767719 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8ea1a-8715-4dca-ad56-92a74b60a826 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.772919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.773644 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.773989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.774268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] PATH_INFO: `/v3/projects/ec41de767f324fb8954e2206383a75e3/users/033a39af2f9f42b1b74002fd229fa067/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.776198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec41de767f324fb8954e2206383a75e3, user_id=033a39af2f9f42b1b74002fd229fa067, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:09.779731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:09.809480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.809480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.809480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.809860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1261/2498] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:09 2022] PUT /identity/v3/projects/ec41de767f324fb8954e2206383a75e3/users/033a39af2f9f42b1b74002fd229fa067/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:09.833846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] MFA Rules not processed for user `033a39af2f9f42b1b74002fd229fa067`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:09.835081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.909928 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1238/2499] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:09.921487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:09.923338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:09.983679 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235377ed-7a39-47b3-8d85-ba67cf93c9af admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:09.987329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:09.987933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:09.988202 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:09.988473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:09.999606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:10.001098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:10.005863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:10.008009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:10.011299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1262/2500] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:10.093298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:10.100543 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21712adf-7e5a-4d02-b2af-31dae5704944 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:10.101936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1e7ea-2217-4697-885c-ea42d36aa89a service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:10.105528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:10.106184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.106470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.106648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:10.106773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] PATH_INFO: `/v3/projects/d994b67a28ac44129851c4d89d71a5e8` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.107233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.107505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.107771 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.107884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d994b67a28ac44129851c4d89d71a5e8)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:10.109575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:10.109967 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:10.159235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd994b67a28ac44129851c4d89d71a5e8', 'request_id': 'req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.165636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd994b67a28ac44129851c4d89d71a5e8', 'request_id': 'req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.173916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd994b67a28ac44129851c4d89d71a5e8', 'request_id': 'req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.176220 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:10.177695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:10.182191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:10.185535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] The token cache is being invalidate because project d994b67a28ac44129851c4d89d71a5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:10.185846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.186539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.187561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1263/2501] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:10.188112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1239/2502] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:10 2022] DELETE /identity/v3/projects/d994b67a28ac44129851c4d89d71a5e8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:10.191978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:10.203374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b404a98-90f6-4759-918b-d0390a41b497 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:10.267024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:10.267844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.268248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.268526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] PATH_INFO: `/v3/users/84496e7152fd41419a37925dfc2f5863` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.269777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] RBAC: Authorizing `identity:delete_user(user_id=84496e7152fd41419a37925dfc2f5863)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:10.271814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:10.325865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84496e7152fd41419a37925dfc2f5863'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.341293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84496e7152fd41419a37925dfc2f5863'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.353488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84496e7152fd41419a37925dfc2f5863'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.354127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84496e7152fd41419a37925dfc2f5863'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.361469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ccff0ed1-b532-410c-bd21-7b533c34990f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84496e7152fd41419a37925dfc2f5863'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.367407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1264/2503] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:10 2022] DELETE /identity/v3/users/84496e7152fd41419a37925dfc2f5863 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:10.650437 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:10.651937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:10.665205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdf781-84d9-4bd6-9e25-d5d2b680b1fb admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:10.677554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:10.732492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:10.733430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.733718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.733996 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] PATH_INFO: `/v3/projects/1fb395c6bb9644318479228593d1a5b6` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.735179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb395c6bb9644318479228593d1a5b6)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:10.737184 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:10.788223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb395c6bb9644318479228593d1a5b6', 'request_id': 'req-91387d37-e312-47b8-b859-0ef49b6a7df8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.794856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb395c6bb9644318479228593d1a5b6', 'request_id': 'req-91387d37-e312-47b8-b859-0ef49b6a7df8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.802557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb395c6bb9644318479228593d1a5b6', 'request_id': 'req-91387d37-e312-47b8-b859-0ef49b6a7df8'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.816658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] The token cache is being invalidate because project 1fb395c6bb9644318479228593d1a5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:10.816977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.817642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-91387d37-e312-47b8-b859-0ef49b6a7df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:10.819141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1240/2504] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:10 2022] DELETE /identity/v3/projects/1fb395c6bb9644318479228593d1a5b6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:10.825807 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.826263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.826629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.847066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:10.848275 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:10.920604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1265/2505] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:10.928376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.928668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.929108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.964295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:10.965975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:10.974997 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1241/2506] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:10.978898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:10.990152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ddc95-41bf-4781-a884-c4c296cd2761 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:10.994801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:10.995831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:10.996362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:10.996712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:10.997601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:10.998020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:10.999656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.013132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1266/2507] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:11.017549 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.028095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1de6b9-78b7-4e37-99df-31204f10c7eb None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.032726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.033361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.033652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.033918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.034550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.036433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.067176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1242/2508] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:11 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:11.068640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.081097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10da41-df5f-4f26-836c-87eaeab848c8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.085542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:11.152323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.153135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.153488 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.153718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.154360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.156294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.186436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1267/2509] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:11 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:11.191467 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.203049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45776ad3-1d4d-4976-9bc5-e57d9eb473a6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.208601 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.209271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.209490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.209762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.210577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.212168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.234330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1243/2510] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:11.236257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.248105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf816-596d-48c4-82f2-dbb0234d6df2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.252837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e1f2b-c032-45a8-9ccf-aa059bf7b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.253433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1f2b-c032-45a8-9ccf-aa059bf7b059 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.253711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1f2b-c032-45a8-9ccf-aa059bf7b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.254005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1f2b-c032-45a8-9ccf-aa059bf7b059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.254637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e1f2b-c032-45a8-9ccf-aa059bf7b059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.256263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e1f2b-c032-45a8-9ccf-aa059bf7b059 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.267938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1268/2511] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:11.272256 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.282782 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456421a-746d-4bbd-a8bb-efbc13397437 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.287757 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.288391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.288665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.288974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] PATH_INFO: `/v3/projects/8767dfb9c0a141d9a0496fcc060f95a0/users/8a1d9205bc984988ae2ae9da2cbf1270/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.303755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8767dfb9c0a141d9a0496fcc060f95a0, user_id=8a1d9205bc984988ae2ae9da2cbf1270, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.306446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.320569 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1244/2512] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:11 2022] PUT /identity/v3/projects/8767dfb9c0a141d9a0496fcc060f95a0/users/8a1d9205bc984988ae2ae9da2cbf1270/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:11.332959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.333238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.333514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.365407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] MFA Rules not processed for user `8a1d9205bc984988ae2ae9da2cbf1270`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:11.366622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:11.437050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1269/2513] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:11.441279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.489229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:11.509022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0bd17-9309-484b-888c-ffcbb221325c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.514686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.515292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.515573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.515843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.516476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.518446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.548568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1245/2514] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:11 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:11.552757 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.564251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061bbb9-01d3-4bc7-8b76-0901135eeea8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.626787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.627435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.627720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.628001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.628796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.630641 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.658428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1270/2515] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:11 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:11.659421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.672525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356bef9-b6a5-492c-9174-160a6d36cba2 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.681449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.682000 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.682378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.684102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.685119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.687116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.699089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1246/2516] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:11.703705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.711117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:11.715631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537730f-f142-4f4c-bd88-66a05ebf7052 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.722105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.723317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.723662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.725353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] PATH_INFO: `/v3/projects/202606198f4e4d94b23132530a114ce1/users/5453f36a0d18491bbc9c9e75efe7c98a/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.744303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202606198f4e4d94b23132530a114ce1, user_id=5453f36a0d18491bbc9c9e75efe7c98a, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.748537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.764346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1271/2517] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:11 2022] PUT /identity/v3/projects/202606198f4e4d94b23132530a114ce1/users/5453f36a0d18491bbc9c9e75efe7c98a/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:11.767482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.777620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc928919-664a-43c7-b352-3da8e2dbda6f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.836574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e32fbe-c356-4988-af18-cf0ffd6c5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.837218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e32fbe-c356-4988-af18-cf0ffd6c5710 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.837496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e32fbe-c356-4988-af18-cf0ffd6c5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.837775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e32fbe-c356-4988-af18-cf0ffd6c5710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.838829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e32fbe-c356-4988-af18-cf0ffd6c5710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.840530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e32fbe-c356-4988-af18-cf0ffd6c5710 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.851176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1247/2518] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:11.856277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:11.867164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ad4a1-2296-4fe3-89f7-c86fd7f60632 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:11.872668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:11.873244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.873514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.873783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] PATH_INFO: `/v3/projects/202606198f4e4d94b23132530a114ce1/users/5453f36a0d18491bbc9c9e75efe7c98a/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.875696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=202606198f4e4d94b23132530a114ce1, user_id=5453f36a0d18491bbc9c9e75efe7c98a, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:11.878519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:11.888889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1272/2519] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:11 2022] PUT /identity/v3/projects/202606198f4e4d94b23132530a114ce1/users/5453f36a0d18491bbc9c9e75efe7c98a/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:11.899049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:11.899331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:11.899605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:11.927510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] MFA Rules not processed for user `5453f36a0d18491bbc9c9e75efe7c98a`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:11.928708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:12.000755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1248/2520] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:12.012134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:12.013906 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:12.074816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabbdff-314f-44a0-9e6e-a07b49a37cbe admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:12.078483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627163-d734-4504-b123-94a2386a610f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:12.079082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627163-d734-4504-b123-94a2386a610f service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:12.079358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627163-d734-4504-b123-94a2386a610f service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:12.079631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627163-d734-4504-b123-94a2386a610f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:12.093041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627163-d734-4504-b123-94a2386a610f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:12.094583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627163-d734-4504-b123-94a2386a610f service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:12.107931 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1273/2521] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:12.154210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:12.155741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:12.169703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2839ba-8727-4d0e-a334-3b2f83da71f4 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:12.174234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811c760-03d1-4081-a90e-767b57d0919d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:12.175053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811c760-03d1-4081-a90e-767b57d0919d service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:12.175328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811c760-03d1-4081-a90e-767b57d0919d service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:12.175592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811c760-03d1-4081-a90e-767b57d0919d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:12.210831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:12.237293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811c760-03d1-4081-a90e-767b57d0919d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:12.238740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811c760-03d1-4081-a90e-767b57d0919d service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:12.248161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1249/2522] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:15.584861 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627163-d734-4504-b123-94a2386a610f service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:15.650587 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627163-d734-4504-b123-94a2386a610f service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:15.655580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:15.656192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:15.656463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:15.656726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] PATH_INFO: `/v3/users/8a1d9205bc984988ae2ae9da2cbf1270` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:15.657952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1d9205bc984988ae2ae9da2cbf1270)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:15.659930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:15.713315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1d9205bc984988ae2ae9da2cbf1270'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:15.721659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1d9205bc984988ae2ae9da2cbf1270'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:15.734816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1d9205bc984988ae2ae9da2cbf1270'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:15.735450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1d9205bc984988ae2ae9da2cbf1270'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:15.736631 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:15.746502 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1d9205bc984988ae2ae9da2cbf1270'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:15.752531 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1274/2523] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:15 2022] DELETE /identity/v3/users/8a1d9205bc984988ae2ae9da2cbf1270 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:15.757004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:15.757493 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:15.757586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:15.776809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:15.778018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:15.838441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1250/2524] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:15.848896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:15.850314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:15.917977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c0281-34c7-4c50-a976-2d41ffcc73d0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:15.922859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:15.923726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:15.924132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:15.924529 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:15.938232 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:15.940117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:15.953810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1275/2525] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:16.239436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:16.242200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:16.276248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f76772-4cc5-47de-bb3b-11d32fb2c03c None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:16.283672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:16.284352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:16.284843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:16.285192 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] PATH_INFO: `/v3/projects/8767dfb9c0a141d9a0496fcc060f95a0` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:16.286388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8767dfb9c0a141d9a0496fcc060f95a0)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:16.288503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:16.351532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8767dfb9c0a141d9a0496fcc060f95a0', 'request_id': 'req-513dcf41-6eac-49df-aac9-a397249b7006'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.359172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8767dfb9c0a141d9a0496fcc060f95a0', 'request_id': 'req-513dcf41-6eac-49df-aac9-a397249b7006'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.368445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8767dfb9c0a141d9a0496fcc060f95a0', 'request_id': 'req-513dcf41-6eac-49df-aac9-a397249b7006'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.383878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] The token cache is being invalidate because project 8767dfb9c0a141d9a0496fcc060f95a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:16.384216 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.385027 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.389943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1251/2526] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:16 2022] DELETE /identity/v3/projects/8767dfb9c0a141d9a0496fcc060f95a0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:16.393843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:16.405198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd7c0-bb91-4454-a161-456a200df2b3 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:16.484528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:16.486083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:16.499208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:16.499795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:16.500042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:16.500311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] PATH_INFO: `/v3/users/5453f36a0d18491bbc9c9e75efe7c98a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:16.501689 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5453f36a0d18491bbc9c9e75efe7c98a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:16.503818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:16.569844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:16.600036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dcf41-6eac-49df-aac9-a397249b7006 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:16.607934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:16.608293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:16.608801 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:16.609131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:16.611277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:16.621727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5453f36a0d18491bbc9c9e75efe7c98a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.630539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:16.630539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:16.637130 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5453f36a0d18491bbc9c9e75efe7c98a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.642790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1252/2527] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:16.661134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5453f36a0d18491bbc9c9e75efe7c98a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.662201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5453f36a0d18491bbc9c9e75efe7c98a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.672439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1204667f-4465-4543-8c63-889ef3ddf42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5453f36a0d18491bbc9c9e75efe7c98a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:16.683816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1276/2528] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:16 2022] DELETE /identity/v3/users/5453f36a0d18491bbc9c9e75efe7c98a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:17.176474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.177992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:17.194201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9f34-ea81-4951-bbed-9477db2002c7 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.252572 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.253144 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.253416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.253682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] PATH_INFO: `/v3/projects/202606198f4e4d94b23132530a114ce1` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.254807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=202606198f4e4d94b23132530a114ce1)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.256722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.304106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202606198f4e4d94b23132530a114ce1', 'request_id': 'req-ac34c143-ccea-482a-a048-c80964e64adc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:17.310297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202606198f4e4d94b23132530a114ce1', 'request_id': 'req-ac34c143-ccea-482a-a048-c80964e64adc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:17.317153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202606198f4e4d94b23132530a114ce1', 'request_id': 'req-ac34c143-ccea-482a-a048-c80964e64adc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:17.328430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] The token cache is being invalidate because project 202606198f4e4d94b23132530a114ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:17.328712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:17.329362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ac34c143-ccea-482a-a048-c80964e64adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:17.330826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1253/2529] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:17 2022] DELETE /identity/v3/projects/202606198f4e4d94b23132530a114ce1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:17.335657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.336022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.336300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.357353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:17.358577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:17.363968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:17.421067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1277/2530] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:17.430139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.430419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.430695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.451160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:17.452369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:17.460140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1254/2531] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:17.465031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.475069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051eed6-b8e3-427d-9991-298cefd0a7e9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.480609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.481244 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.481524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.481794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.482437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:17.482738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.484478 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.496745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1278/2532] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:17.501038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.510472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae35161-2475-4c77-8952-046b697048f3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.515069 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.515665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.515948 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.516222 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.516881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.518791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.547886 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1255/2533] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:17 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:17.552118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.562038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435b5f8-401a-41ee-8b93-db93c024c037 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.618427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.619024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.619300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.619573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.620406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.622574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.643687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1279/2534] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:17 2022] POST /identity/v3/users => generated 305 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:17.647727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.658756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b8453-4f8b-4eeb-9e99-952aeb955cb5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.663374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.663962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.664241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.664515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.665195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.666835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.676704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1256/2535] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:17.680550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.690832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285885-3b8f-432e-96aa-ef409c34b6f3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.696088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bb7ff-1b5d-4052-ace4-ab1c4675885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.696678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bb7ff-1b5d-4052-ace4-ab1c4675885f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.697010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bb7ff-1b5d-4052-ace4-ab1c4675885f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.697278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bb7ff-1b5d-4052-ace4-ab1c4675885f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.698446 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bb7ff-1b5d-4052-ace4-ab1c4675885f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.700119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bb7ff-1b5d-4052-ace4-ab1c4675885f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.710707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1280/2536] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:17.715644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.726112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632633f5-245f-42f0-b357-7e8f5fa1437c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.732137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.732845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.733122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.733509 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] PATH_INFO: `/v3/projects/f500d1c521dd47ed8d9e02fc7ecfd0cf/users/f4a1980f04774e7597b9062f148569b4/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.749575 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f500d1c521dd47ed8d9e02fc7ecfd0cf, user_id=f4a1980f04774e7597b9062f148569b4, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.752290 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:17.766526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1257/2537] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:17 2022] PUT /identity/v3/projects/f500d1c521dd47ed8d9e02fc7ecfd0cf/users/f4a1980f04774e7597b9062f148569b4/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:17.777850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.778133 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.778403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.808932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] MFA Rules not processed for user `f4a1980f04774e7597b9062f148569b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:17.808932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:17.884663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1281/2538] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:17 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:17.886328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:17.912790 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:17.973056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767a617-03a4-4174-89cf-c4ce1f9c533a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:17.979573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:17.980404 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:17.980803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:17.981342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:17.981682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:17.983605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.016705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1258/2539] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:17 2022] POST /identity/v3/projects => generated 409 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:18.020735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.032842 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f7c1f-f4b2-4463-8a7d-164edfbc265c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.036943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:18.115554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.116171 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.116450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.116722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.117466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.119499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.154249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1282/2540] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:18 2022] POST /identity/v3/users => generated 311 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:18.158239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.171521 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19e6a-806e-4aa9-8292-92fcf7d9e403 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.185781 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.186500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.186676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.186951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.187606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.189284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.200937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1259/2541] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:18.205289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.216934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed2fba-3c2e-4981-929e-c04ac9de6fbf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.221790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.222522 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.222903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.223238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] PATH_INFO: `/v3/projects/e0624a0cc9694db09186364fd86fd49c/users/225f2da5d3fd402eb2c2b897126ea63a/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.240138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0624a0cc9694db09186364fd86fd49c, user_id=225f2da5d3fd402eb2c2b897126ea63a, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.243303 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.256137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1283/2542] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:18 2022] PUT /identity/v3/projects/e0624a0cc9694db09186364fd86fd49c/users/225f2da5d3fd402eb2c2b897126ea63a/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:18.260716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.271013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa59c90-7aa2-400f-a8eb-e82a5416e8e6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.334088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09421470-4692-4a68-8364-18da99032646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.334702 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421470-4692-4a68-8364-18da99032646 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.335019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421470-4692-4a68-8364-18da99032646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.335324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421470-4692-4a68-8364-18da99032646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.335988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09421470-4692-4a68-8364-18da99032646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.337663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09421470-4692-4a68-8364-18da99032646 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.349484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1260/2543] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:18.354445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.366702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb301d2f-434d-4b13-a658-01906f613928 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.372524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.373196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.373516 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.373794 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] PATH_INFO: `/v3/projects/e0624a0cc9694db09186364fd86fd49c/users/225f2da5d3fd402eb2c2b897126ea63a/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.375733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0624a0cc9694db09186364fd86fd49c, user_id=225f2da5d3fd402eb2c2b897126ea63a, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.378544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.391069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1284/2544] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:18 2022] PUT /identity/v3/projects/e0624a0cc9694db09186364fd86fd49c/users/225f2da5d3fd402eb2c2b897126ea63a/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:18.403622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.403909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.404235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.434164 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] MFA Rules not processed for user `225f2da5d3fd402eb2c2b897126ea63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:18.435800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:18.509231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1261/2545] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:18 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:18.520378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.522169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:18.588335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4af9a-11bc-4e2b-ab87-71e1cf74b15e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.592837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.593440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.593718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.593987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.602964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.604511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.613863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1285/2546] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:18 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:18.775853 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:18.785334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc17ae5-f8a3-492a-9fad-c3d20d243da3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:18.788748 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48ee1-8114-432b-a89a-8243ffb23fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:18.789446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48ee1-8114-432b-a89a-8243ffb23fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:18.789716 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48ee1-8114-432b-a89a-8243ffb23fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:18.790858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48ee1-8114-432b-a89a-8243ffb23fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:18.795061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:18.849896 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c48ee1-8114-432b-a89a-8243ffb23fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:18.851327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c48ee1-8114-432b-a89a-8243ffb23fb5 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:18.860397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1262/2547] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:19.356384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:19.357880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:19.425567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941fbe1-2183-4451-a564-b9e994f8565f service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:19.430337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:19.431139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:19.431374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:19.431810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] PATH_INFO: `/v3/users/08c3e6dcb2e34ce6b54811134ef115c5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:19.433853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c3e6dcb2e34ce6b54811134ef115c5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:19.436141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:19.439453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:19.486821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c3e6dcb2e34ce6b54811134ef115c5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:19.494495 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c3e6dcb2e34ce6b54811134ef115c5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:19.507594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c3e6dcb2e34ce6b54811134ef115c5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:19.508256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c3e6dcb2e34ce6b54811134ef115c5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:19.515793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c3e6dcb2e34ce6b54811134ef115c5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:19.524987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1286/2548] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:19 2022] DELETE /identity/v3/users/08c3e6dcb2e34ce6b54811134ef115c5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:19.531195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:19.531712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:19.531821 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:19.553771 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:19.555036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:19.613270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:19.614650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:19.622741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1263/2549] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:19.632247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:19.633606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:19.675459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c16cc-2b30-4d6b-816a-25f57d35ea35 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:19.679715 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:19.680314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:19.680588 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:19.680885 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:19.683290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:19.690489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05271e70-d6f2-4d46-8c6a-9c500d492e38 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:19.694114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:19.694755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:19.694976 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:19.695845 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:19.705585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:19.707050 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:19.718150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1264/2550] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:19.747902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:19.749434 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:19.759213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1287/2551] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:20.039730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:20.042800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:20.068296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf48a9-d8ba-4c33-8aca-973c987c81a3 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:20.073153 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:20.073856 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:20.074132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:20.074400 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] PATH_INFO: `/v3/projects/bffc258aa06a454c83849dc6ff324518` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:20.075910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bffc258aa06a454c83849dc6ff324518)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:20.077860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:20.122267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:20.127075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bffc258aa06a454c83849dc6ff324518', 'request_id': 'req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.134959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bffc258aa06a454c83849dc6ff324518', 'request_id': 'req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.143111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bffc258aa06a454c83849dc6ff324518', 'request_id': 'req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.155424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] The token cache is being invalidate because project bffc258aa06a454c83849dc6ff324518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:20.155717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.156369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.157891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1265/2552] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:20 2022] DELETE /identity/v3/projects/bffc258aa06a454c83849dc6ff324518 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:20.161892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:20.175528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc9c7b-8d46-4829-86c9-5ea72a7817d8 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:20.212449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:20.259262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:20.260186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:20.260720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:20.261212 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] PATH_INFO: `/v3/users/033a39af2f9f42b1b74002fd229fa067` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:20.262863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=033a39af2f9f42b1b74002fd229fa067)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:20.265802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:20.342381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '033a39af2f9f42b1b74002fd229fa067'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.350405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '033a39af2f9f42b1b74002fd229fa067'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.368723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '033a39af2f9f42b1b74002fd229fa067'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.370796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '033a39af2f9f42b1b74002fd229fa067'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.382682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-add08e12-0ecd-4378-b8fe-d8cc60cffa8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '033a39af2f9f42b1b74002fd229fa067'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.390322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1288/2553] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:20 2022] DELETE /identity/v3/users/033a39af2f9f42b1b74002fd229fa067 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:20.705568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:20.707118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:20.724747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2d76e-67ea-49eb-b930-9f35e0f2e0e3 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:20.798115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:20.798751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:20.799161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:20.799441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] PATH_INFO: `/v3/projects/ec41de767f324fb8954e2206383a75e3` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:20.800566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec41de767f324fb8954e2206383a75e3)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:20.802555 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:20.860228 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec41de767f324fb8954e2206383a75e3', 'request_id': 'req-8a2ccf61-8a81-4921-aef0-5c5d857f715a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.868127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec41de767f324fb8954e2206383a75e3', 'request_id': 'req-8a2ccf61-8a81-4921-aef0-5c5d857f715a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.878243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec41de767f324fb8954e2206383a75e3', 'request_id': 'req-8a2ccf61-8a81-4921-aef0-5c5d857f715a'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.889936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] The token cache is being invalidate because project ec41de767f324fb8954e2206383a75e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:20.890309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.891278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-8a2ccf61-8a81-4921-aef0-5c5d857f715a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:20.893132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1266/2554] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:20 2022] DELETE /identity/v3/projects/ec41de767f324fb8954e2206383a75e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:20.897739 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:20.898025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:20.898296 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:20.923484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:20.924716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:20.987820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1289/2555] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:20.996989 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:20.997282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:20.997733 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.019841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:21.021111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:21.029965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1267/2556] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:21.034059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.044609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b65758-1455-4011-9a30-60b3e492d91c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.049854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.051092 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.051367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.051636 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.052282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:21.052577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.054225 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.071050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1290/2557] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:36:21.077659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.087054 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6752a5-9f00-4407-a465-5ca4aa4d55ff None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.092310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.093062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.093346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.093616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.094242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.096156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.129885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1268/2558] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:21 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:21.131044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.142423 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24606798-701b-431d-beed-63fd5e0d625e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.179260 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:21.206209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.206835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.207099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.207400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.208026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.210006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.231585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1291/2559] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:21 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:21.235914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.246860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fad3c-98c1-4ca5-bec1-ed50d5a43b88 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.251881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.252491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.252795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.253077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.253725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.255362 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.267213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1269/2560] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:21.271239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.282274 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406aa94-a2f2-4c36-8da7-dfed2c60390b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.287159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9593714-ab66-4de9-bda0-16c9f9e63c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.287766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9593714-ab66-4de9-bda0-16c9f9e63c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.288046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9593714-ab66-4de9-bda0-16c9f9e63c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.288324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9593714-ab66-4de9-bda0-16c9f9e63c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.289019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9593714-ab66-4de9-bda0-16c9f9e63c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.290713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9593714-ab66-4de9-bda0-16c9f9e63c55 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.301500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1292/2561] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:21.305816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.317405 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404667e2-d2ad-43a4-935d-3b5696006fcc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.322090 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.322787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.323874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.324166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] PATH_INFO: `/v3/projects/486a79566e3b4007943db7d91f09b280/users/583d242d3d614574ad251e209c430014/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.339212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486a79566e3b4007943db7d91f09b280, user_id=583d242d3d614574ad251e209c430014, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.342202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.350452 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:21.357873 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1270/2562] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:21 2022] PUT /identity/v3/projects/486a79566e3b4007943db7d91f09b280/users/583d242d3d614574ad251e209c430014/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:21.368348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.368721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.369091 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.398867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] MFA Rules not processed for user `583d242d3d614574ad251e209c430014`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:21.400286 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:21.478378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1293/2563] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:21.483016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.564463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.565603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baee28-35e7-4d9d-be6b-9b9e9161b800 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.566044 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:21.570743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.571439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.571780 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.572117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.572908 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.575302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.613286 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1271/2564] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:21 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:21.616938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.664948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f42e23-8499-4381-9c2a-b3e4c2c02e1d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.711897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755af3e-b3fa-4a2e-b6c0-d90c25859ff5 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.720162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.721483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.722515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.723011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.724953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.726960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.736122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:21.754122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1272/2565] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:21 2022] POST /identity/v3/users => generated 301 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:21.757736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.765581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.766566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.767002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.767518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.769554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:21.770700 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28439a6-c352-4864-9979-ed1121431d4e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.777052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.777745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.778088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.778422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.779744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.781590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.784871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.786662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.795129 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1273/2566] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:21.799533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1294/2567] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:21.799715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.815961 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff269990-b1b7-4cdf-ac83-4b7d4783a3ec admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.822032 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.823470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.823911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.824305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] PATH_INFO: `/v3/projects/d4980537f17d469b96d2b643af0e603c/users/f4de37aee82e44a3bbc8b066e99875c2/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.841213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4980537f17d469b96d2b643af0e603c, user_id=f4de37aee82e44a3bbc8b066e99875c2, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.844201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.858537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1274/2568] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:21 2022] PUT /identity/v3/projects/d4980537f17d469b96d2b643af0e603c/users/f4de37aee82e44a3bbc8b066e99875c2/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:21.862718 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.873537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7747-5bce-404c-b8eb-c3be42d42f84 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.934149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96271ec7-59ed-411c-a1e9-b972f793dad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.934744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271ec7-59ed-411c-a1e9-b972f793dad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.935469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271ec7-59ed-411c-a1e9-b972f793dad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.935469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271ec7-59ed-411c-a1e9-b972f793dad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.935896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96271ec7-59ed-411c-a1e9-b972f793dad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.937519 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96271ec7-59ed-411c-a1e9-b972f793dad0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.947696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1295/2569] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:21.952100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:21.963589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923bf7e-9638-4753-8cab-05e3e08d1cf9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:21.968645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:21.969335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.969617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.969891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] PATH_INFO: `/v3/projects/d4980537f17d469b96d2b643af0e603c/users/f4de37aee82e44a3bbc8b066e99875c2/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:21.971933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4980537f17d469b96d2b643af0e603c, user_id=f4de37aee82e44a3bbc8b066e99875c2, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:21.974708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:21.985952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1275/2570] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:21 2022] PUT /identity/v3/projects/d4980537f17d469b96d2b643af0e603c/users/f4de37aee82e44a3bbc8b066e99875c2/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:21.997950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:21.998311 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:21.998653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:22.027277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] MFA Rules not processed for user `f4de37aee82e44a3bbc8b066e99875c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:22.028420 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:22.101597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1296/2571] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:22.112794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:22.114519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:22.143855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:22.174377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557088fd-dcb4-4990-b7a3-894cf4a7bef7 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:22.177852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:22.178449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:22.178722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:22.178995 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:22.188841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:22.190276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:22.199722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1276/2572] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:22.247234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:22.248628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:22.260391 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0972cd8-c225-488f-8496-6e70a1e4c8e3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:22.264628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec519f-f75b-40c6-a4ec-d4c9125059dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:22.265293 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec519f-f75b-40c6-a4ec-d4c9125059dd service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:22.265571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec519f-f75b-40c6-a4ec-d4c9125059dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:22.265840 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec519f-f75b-40c6-a4ec-d4c9125059dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:22.314387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:22.324513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec519f-f75b-40c6-a4ec-d4c9125059dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:22.326015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec519f-f75b-40c6-a4ec-d4c9125059dd service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:22.335370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1297/2573] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:25.818608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.881951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e471-f857-49c9-96fb-414c158f0923 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.886930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.887557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.887834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.888106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] PATH_INFO: `/v3/users/583d242d3d614574ad251e209c430014` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.889260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=583d242d3d614574ad251e209c430014)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.891186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.940138 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583d242d3d614574ad251e209c430014'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:25.947212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583d242d3d614574ad251e209c430014'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:25.947959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583d242d3d614574ad251e209c430014'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:25.954559 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583d242d3d614574ad251e209c430014'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:25.967406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583d242d3d614574ad251e209c430014'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:25.973325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1277/2574] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:25 2022] DELETE /identity/v3/users/583d242d3d614574ad251e209c430014 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:25.977698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.978138 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.978233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.998190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:25.999401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.059609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1298/2575] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:26.069077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.072794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.126647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:26.132753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cf8e8-e809-4ed0-a680-091987f851e8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.137234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.137848 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.138175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.138624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.147640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.149121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.158898 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1278/2576] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:26.426195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.428064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.438544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.440354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.444490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb2a8e-ca59-4d62-af25-f94a307cc614 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.452256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.452998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.453357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.453692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] PATH_INFO: `/v3/users/f4a1980f04774e7597b9062f148569b4` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.454926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4a1980f04774e7597b9062f148569b4)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.456915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.461131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2747-a4e7-4826-b198-6e8df47cfd07 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.480883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.481663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.481947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.482224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] PATH_INFO: `/v3/projects/486a79566e3b4007943db7d91f09b280` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.483755 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486a79566e3b4007943db7d91f09b280)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.488002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.519707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4a1980f04774e7597b9062f148569b4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.530042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4a1980f04774e7597b9062f148569b4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.543942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486a79566e3b4007943db7d91f09b280', 'request_id': 'req-a6eca8f1-4d16-471b-baf3-146f662361e2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.545604 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4a1980f04774e7597b9062f148569b4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.546347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4a1980f04774e7597b9062f148569b4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.552329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486a79566e3b4007943db7d91f09b280', 'request_id': 'req-a6eca8f1-4d16-471b-baf3-146f662361e2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.554584 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-869ef708-6b00-439c-88b6-b8ddf553624a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4a1980f04774e7597b9062f148569b4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.562476 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486a79566e3b4007943db7d91f09b280', 'request_id': 'req-a6eca8f1-4d16-471b-baf3-146f662361e2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.563193 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1299/2577] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:26 2022] DELETE /identity/v3/users/f4a1980f04774e7597b9062f148569b4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:26.567328 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.567686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.568179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.578024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] The token cache is being invalidate because project 486a79566e3b4007943db7d91f09b280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:26.578615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.579340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.583933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1279/2578] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:26 2022] DELETE /identity/v3/projects/486a79566e3b4007943db7d91f09b280 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:26.585159 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.586574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.588126 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:26.589622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.654675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1300/2579] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:26.660282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca8f1-4d16-471b-baf3-146f662361e2 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.664702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.665053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.665687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.665966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.666155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.666249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] PATH_INFO: `/v3/users/f4de37aee82e44a3bbc8b066e99875c2` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.667915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4de37aee82e44a3bbc8b066e99875c2)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.669891 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.671792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:26.718919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4de37aee82e44a3bbc8b066e99875c2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.725824 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4de37aee82e44a3bbc8b066e99875c2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.726490 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4de37aee82e44a3bbc8b066e99875c2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.731161 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dd3c-6591-4925-9c5d-b566d225a937 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.734893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4de37aee82e44a3bbc8b066e99875c2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.735046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.735585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.735866 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.736137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.737936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.743583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:26.750023 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4de37aee82e44a3bbc8b066e99875c2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:26.750623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.752049 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.757420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1280/2580] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:26 2022] DELETE /identity/v3/users/f4de37aee82e44a3bbc8b066e99875c2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:26.761878 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1301/2581] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:27.122855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.124453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.167821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.169580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.186869 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4450df-1a75-4640-877d-31ba58196225 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.210609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ce00-25fc-42aa-8402-3f362257ecb6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.215315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.215951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.216236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.217093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] PATH_INFO: `/v3/projects/d4980537f17d469b96d2b643af0e603c` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.217927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4980537f17d469b96d2b643af0e603c)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.219862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.251316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.251848 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.252120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.252397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] PATH_INFO: `/v3/projects/f500d1c521dd47ed8d9e02fc7ecfd0cf` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.253559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f500d1c521dd47ed8d9e02fc7ecfd0cf)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.255468 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.269223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4980537f17d469b96d2b643af0e603c', 'request_id': 'req-2b8b3bc8-4bba-418c-b287-90b699df4602'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.275318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4980537f17d469b96d2b643af0e603c', 'request_id': 'req-2b8b3bc8-4bba-418c-b287-90b699df4602'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.283525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4980537f17d469b96d2b643af0e603c', 'request_id': 'req-2b8b3bc8-4bba-418c-b287-90b699df4602'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.295318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] The token cache is being invalidate because project d4980537f17d469b96d2b643af0e603c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:27.295708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.296353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2b8b3bc8-4bba-418c-b287-90b699df4602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.297938 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1281/2582] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:27 2022] DELETE /identity/v3/projects/d4980537f17d469b96d2b643af0e603c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:27.303304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a6c0e-ae02-4655-bcd4-a8e65a433202 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.303585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a6c0e-ae02-4655-bcd4-a8e65a433202 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.303857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a6c0e-ae02-4655-bcd4-a8e65a433202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.306572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f500d1c521dd47ed8d9e02fc7ecfd0cf', 'request_id': 'req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.313613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f500d1c521dd47ed8d9e02fc7ecfd0cf', 'request_id': 'req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.322281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f500d1c521dd47ed8d9e02fc7ecfd0cf', 'request_id': 'req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.324839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a93a6c0e-ae02-4655-bcd4-a8e65a433202 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:27.326044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a93a6c0e-ae02-4655-bcd4-a8e65a433202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.334859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] The token cache is being invalidate because project f500d1c521dd47ed8d9e02fc7ecfd0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:27.335255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.335959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.337493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1302/2583] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:27 2022] DELETE /identity/v3/projects/f500d1c521dd47ed8d9e02fc7ecfd0cf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:27.341271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.342810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.383966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:27.389560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1282/2584] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:27.400157 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.400463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.401122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.419844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf89d83-2cda-43ce-b7e4-0f0ef9bc1479 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.424521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.425040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:27.425176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.425451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.425721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] PATH_INFO: `/v3/users/225f2da5d3fd402eb2c2b897126ea63a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.426536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.426853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=225f2da5d3fd402eb2c2b897126ea63a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.428844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.441110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1283/2585] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:27.442721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.455754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab2a9d-2658-465f-8cf1-54fa9486b448 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.461469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.462121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.462468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.462800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.463651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:27.463651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.465695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.478063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1284/2586] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:36:27.480750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '225f2da5d3fd402eb2c2b897126ea63a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.483597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.488188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '225f2da5d3fd402eb2c2b897126ea63a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.488487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:27.496088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed63d4-861d-4617-b316-e3bce458a19a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.502119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '225f2da5d3fd402eb2c2b897126ea63a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.502750 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '225f2da5d3fd402eb2c2b897126ea63a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.502876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.503435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.503698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.503972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.504597 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.506528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.510289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '225f2da5d3fd402eb2c2b897126ea63a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:27.516512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1303/2587] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:27 2022] DELETE /identity/v3/users/225f2da5d3fd402eb2c2b897126ea63a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:27.537988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1285/2588] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:27.538773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.540308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.610111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4b5bb-8862-4e42-a38b-873fbbc43f25 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.682974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.683608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.683876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.684149 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.684916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.687194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.696697 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:27.714707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1304/2589] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:27.718407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.730293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44dbf5-a673-4596-be3c-d49cda7cf16b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.735839 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.736556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.736897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.737183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.737883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.740111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.752501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1286/2590] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:27.757298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.772666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b762883-189b-498e-8a76-0bebb03e39eb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.777336 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0393-057a-4d70-995c-bdccbddab2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.778027 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0393-057a-4d70-995c-bdccbddab2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.778813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0393-057a-4d70-995c-bdccbddab2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.778813 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0393-057a-4d70-995c-bdccbddab2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.779416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0393-057a-4d70-995c-bdccbddab2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.781175 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0393-057a-4d70-995c-bdccbddab2b5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.796804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1305/2591] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:27.799519 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.810582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e648ee-2ca0-4f83-8023-e807861a9c35 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.815231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.815878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.816154 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.816427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] PATH_INFO: `/v3/projects/3e5804256bf54a34987a945e1a5fd462/users/cff0fd1006504b97ae015ac9aaca050b/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.832187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5804256bf54a34987a945e1a5fd462, user_id=cff0fd1006504b97ae015ac9aaca050b, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.834905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.849312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1287/2592] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:27 2022] PUT /identity/v3/projects/3e5804256bf54a34987a945e1a5fd462/users/cff0fd1006504b97ae015ac9aaca050b/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:27.859972 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.860407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.860693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.899384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] MFA Rules not processed for user `cff0fd1006504b97ae015ac9aaca050b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:27.900594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.905376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.910117 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.984893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247fa7f-0e4b-4f24-a0f3-9d7333f30c98 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.988407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1306/2593] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:27.989433 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.990052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.990326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.990594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] PATH_INFO: `/v3/projects/e0624a0cc9694db09186364fd86fd49c` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.991705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0624a0cc9694db09186364fd86fd49c)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.992966 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.993659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.004209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3ee4-f869-46c1-8c09-77b777b6bbdd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.008786 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.009383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.009663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.009937 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.011222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.013207 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.041489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1307/2594] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:28.045426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0624a0cc9694db09186364fd86fd49c', 'request_id': 'req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:28.045831 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.051768 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0624a0cc9694db09186364fd86fd49c', 'request_id': 'req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:28.060897 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0624a0cc9694db09186364fd86fd49c', 'request_id': 'req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:28.073281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] The token cache is being invalidate because project e0624a0cc9694db09186364fd86fd49c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:28.073834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:28.074683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4863d2a5-98fb-498c-a45d-d8cc68f1fea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:28.076195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1288/2595] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:27 2022] DELETE /identity/v3/projects/e0624a0cc9694db09186364fd86fd49c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:28.081218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac5d2f-85ed-4860-808f-3c56eff3d414 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.081522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac5d2f-85ed-4860-808f-3c56eff3d414 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.081794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac5d2f-85ed-4860-808f-3c56eff3d414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.101079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-77ac5d2f-85ed-4860-808f-3c56eff3d414 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:28.102288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-77ac5d2f-85ed-4860-808f-3c56eff3d414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.111378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1289/2596] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:28.111520 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d053-9201-4d55-8b33-522ddb6bb55c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.116095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.116728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.117046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.117317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.117934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.120482 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.121397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.121679 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.121980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.142024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:28.143156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1308/2597] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:28.146354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.147289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.148912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.151516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1290/2598] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:28.155740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.161374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44962209-df8e-4084-97cf-09bfba61e796 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.164917 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b610cf9-36fc-413e-87e8-cec797437a69 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.165959 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.167065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.167459 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.167732 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.168424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.170042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.170225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.170803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.171080 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.171345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.171974 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:28.172271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.173866 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.180317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1309/2599] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:28.185093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.186440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1291/2600] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:36:28.190462 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.194652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a9611-8a6e-461a-b262-fe2582746f12 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.199695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.200316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.200595 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.200907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.201556 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.201658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5831-6730-4954-ab40-e139727ff030 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.203139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.206468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.207049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.207326 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.207598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.208212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.210133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.214101 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1310/2601] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:28.215875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.218702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.229580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2892829-bdb9-4856-810f-b730976da628 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.235011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a701de-179b-40dd-b932-d6e7bf204056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.235767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a701de-179b-40dd-b932-d6e7bf204056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.236109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a701de-179b-40dd-b932-d6e7bf204056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.236485 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a701de-179b-40dd-b932-d6e7bf204056 admin admin] PATH_INFO: `/v3/projects/f274225dc2854f8aa648c8aa1bcfb5b3/users/a1fb1f37d6c64c57b00b723620264432/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.241289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1292/2602] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:28.246605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.249412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.253463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a701de-179b-40dd-b932-d6e7bf204056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f274225dc2854f8aa648c8aa1bcfb5b3, user_id=a1fb1f37d6c64c57b00b723620264432, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.256512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a701de-179b-40dd-b932-d6e7bf204056 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.270205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1311/2603] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:28 2022] PUT /identity/v3/projects/f274225dc2854f8aa648c8aa1bcfb5b3/users/a1fb1f37d6c64c57b00b723620264432/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:28.281888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.282170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.282445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.309492 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923276-d9fd-4f07-905e-1c232d3da39c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.311272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] MFA Rules not processed for user `a1fb1f37d6c64c57b00b723620264432`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:28.312492 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.369313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.370489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.370759 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.371025 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.371639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.373580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.378901 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1312/2604] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:28.383787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.394968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1293/2605] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:28.395635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165e9ee-25f4-49e2-9a32-4889eaa80398 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.398796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.400493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.401376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.401823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.402098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.402713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.404623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.408951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4b4f-f357-431a-b926-3d9e6921d83f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.413900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.414486 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.414764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.415036 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.415705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.417844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.431188 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1294/2606] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:28.433420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.439200 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1313/2607] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:28.440245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.445526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876745b-bffe-4448-8176-b1550241ac46 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.504758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.505397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.505677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.505950 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.506652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.507705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c25f44-5950-4bcc-9a6e-ec9a18dee349 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.508200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.512566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.513190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.513474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.513747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.514362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.516276 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.518639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1295/2608] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:28.522794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.533241 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8e69a-e0c8-4d84-b821-d5ff9f52f2a1 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.538834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1314/2609] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:28.543354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.544237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732b0c-cb40-4cb2-bc7a-950597250dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.544968 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732b0c-cb40-4cb2-bc7a-950597250dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.545251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732b0c-cb40-4cb2-bc7a-950597250dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.545528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732b0c-cb40-4cb2-bc7a-950597250dd1 admin admin] PATH_INFO: `/v3/projects/b9fef62fd5894258b1afc5a360472400/users/35bdba197af14f678e6a2a9b3aa635f2/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.549002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.554357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea64c6-6ec6-4cb8-ac29-67d4e7200129 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.559252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.559844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.560117 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.560388 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.561048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.562752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.564223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06732b0c-cb40-4cb2-bc7a-950597250dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fef62fd5894258b1afc5a360472400, user_id=35bdba197af14f678e6a2a9b3aa635f2, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.566932 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06732b0c-cb40-4cb2-bc7a-950597250dd1 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.572967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1315/2610] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:28.577395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.580582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1296/2611] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:28 2022] PUT /identity/v3/projects/b9fef62fd5894258b1afc5a360472400/users/35bdba197af14f678e6a2a9b3aa635f2/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:28.590037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b891b4-2634-414b-b54d-70463e70feb1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.591867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898696-1ae1-4621-a388-149407739203 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.592638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898696-1ae1-4621-a388-149407739203 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.593016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898696-1ae1-4621-a388-149407739203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.626987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.627227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-70898696-1ae1-4621-a388-149407739203 None None] MFA Rules not processed for user `35bdba197af14f678e6a2a9b3aa635f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:28.628571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-70898696-1ae1-4621-a388-149407739203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.653854 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.654619 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.654961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.655239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] PATH_INFO: `/v3/projects/09049ee1f9d141198d6e3d5669850df8/users/560c548ce3b142948bd273fa0e34b1dc/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.670865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09049ee1f9d141198d6e3d5669850df8, user_id=560c548ce3b142948bd273fa0e34b1dc, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.673606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.687451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1316/2612] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:28 2022] PUT /identity/v3/projects/09049ee1f9d141198d6e3d5669850df8/users/560c548ce3b142948bd273fa0e34b1dc/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:28.691289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.698288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1297/2613] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:28.708645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70898696-1ae1-4621-a388-149407739203 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.710439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-70898696-1ae1-4621-a388-149407739203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.757837 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eae73-c424-4fa3-b5dd-94898b99e92c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.762791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.763400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.763676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.763947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.764731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.766379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.767356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70898696-1ae1-4621-a388-149407739203 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.777325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1317/2614] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:28.781396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.790801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b3aa-c3da-4271-9283-494525647411 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.795980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b2d2c-5050-4327-af81-07fb2907f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.796684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b2d2c-5050-4327-af81-07fb2907f4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.796992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b2d2c-5050-4327-af81-07fb2907f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.797308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b2d2c-5050-4327-af81-07fb2907f4e5 admin admin] PATH_INFO: `/v3/projects/09049ee1f9d141198d6e3d5669850df8/users/560c548ce3b142948bd273fa0e34b1dc/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.799997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b2d2c-5050-4327-af81-07fb2907f4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09049ee1f9d141198d6e3d5669850df8, user_id=560c548ce3b142948bd273fa0e34b1dc, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.802758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b2d2c-5050-4327-af81-07fb2907f4e5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.815789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1318/2615] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:28 2022] PUT /identity/v3/projects/09049ee1f9d141198d6e3d5669850df8/users/560c548ce3b142948bd273fa0e34b1dc/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:28.822773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.823953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.824229 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.824497 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.827725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.828085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.828432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.857473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] MFA Rules not processed for user `560c548ce3b142948bd273fa0e34b1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:28.858681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.875660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.889288 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.890904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.901522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1298/2616] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:28.941262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1319/2617] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:28.953855 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.955302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.966911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8695df3-8c0f-418c-90dd-44ce7f2a2093 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:29.028719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214804d-c3ab-4c0b-a9b5-0f9d75221491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:29.029488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214804d-c3ab-4c0b-a9b5-0f9d75221491 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.029773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214804d-c3ab-4c0b-a9b5-0f9d75221491 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.030045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214804d-c3ab-4c0b-a9b5-0f9d75221491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.040204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214804d-c3ab-4c0b-a9b5-0f9d75221491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:29.041933 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214804d-c3ab-4c0b-a9b5-0f9d75221491 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.052750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1299/2618] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:29.109713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:29.119635 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b1255-76f2-4ff9-b223-5f0b1387b58d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:29.123402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:29.123994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.124270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.124540 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.126335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.156583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:29.194113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:29.195723 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.208160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1320/2619] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:30.982804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:30.983340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:30.983444 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:31.005718 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:31.007279 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:31.081218 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1300/2620] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:31.133678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:31.135118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:31.204457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e97dd-4ce9-45cc-8a24-6594ada7820e service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:31.209481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:31.210018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:31.210291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:31.210561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:31.218949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:31.220527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:31.228405 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1321/2621] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:36:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:36:32.107514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.210800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b31f-4f9b-4d22-bdbe-ed3ddedf08c1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:32.218394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:32.218394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.218394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.218680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] PATH_INFO: `/v3/users/35bdba197af14f678e6a2a9b3aa635f2` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.219846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35bdba197af14f678e6a2a9b3aa635f2)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:32.222019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:32.241142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.258083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1d476-29a7-4c1a-9c9d-a01317fcd96a service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:32.263496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:32.264197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.264551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.265136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] PATH_INFO: `/v3/users/cff0fd1006504b97ae015ac9aaca050b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.266753 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff0fd1006504b97ae015ac9aaca050b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:32.273526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:32.297660 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:32.322675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35bdba197af14f678e6a2a9b3aa635f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.332427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35bdba197af14f678e6a2a9b3aa635f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.333501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35bdba197af14f678e6a2a9b3aa635f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.347132 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35bdba197af14f678e6a2a9b3aa635f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.359518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff0fd1006504b97ae015ac9aaca050b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.367202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2cbef1c4-fcce-44cd-abed-da6795760ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35bdba197af14f678e6a2a9b3aa635f2'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.374183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff0fd1006504b97ae015ac9aaca050b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.379843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1301/2622] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:32 2022] DELETE /identity/v3/users/35bdba197af14f678e6a2a9b3aa635f2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:32.385472 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.385844 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.386200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.404135 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff0fd1006504b97ae015ac9aaca050b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.404921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff0fd1006504b97ae015ac9aaca050b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.418309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:32.420119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.429995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ef78e5db-17a7-4abc-973c-72d524d795e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff0fd1006504b97ae015ac9aaca050b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:32.450743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1322/2623] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:32 2022] DELETE /identity/v3/users/cff0fd1006504b97ae015ac9aaca050b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:32.458602 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.458916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.459194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.519310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1302/2624] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:32.525258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:32.529443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.531813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.532626 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.636203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:32.687277 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa77cb-559d-4ddb-9860-c4c11d833cc0 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:32.736821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1323/2625] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:32.738060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.774798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:32.775901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.775901 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.776441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.788306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:32.789872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:32.801169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1303/2626] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:32.802128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.803391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.818777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1b206-c46b-4dfc-8354-3bd8c5099c83 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:32.823470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:32.824074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.824352 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.824626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.835827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:32.839076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:32.843860 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:32.849254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498ac0d-1e11-453b-b871-4fd05f997ead None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:32.858452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:32.859993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.860507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.860938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.864428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.866051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1304/2627] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:33.040680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.042754 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.056265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1324/2628] 10.222.0.81 () {60 vars in 1396 bytes} [Wed Aug 24 03:36:32 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:33.308166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:33.309689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.326532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2538aa2-5e65-4754-bdf1-9b69aa53b3db service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:33.342294 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:33.342931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.343235 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.343468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] PATH_INFO: `/v3/projects/3e5804256bf54a34987a945e1a5fd462` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.353656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5804256bf54a34987a945e1a5fd462)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.356402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.451334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5804256bf54a34987a945e1a5fd462', 'request_id': 'req-e26d23d9-c581-42f8-b5b3-cb6231073aad'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.459441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5804256bf54a34987a945e1a5fd462', 'request_id': 'req-e26d23d9-c581-42f8-b5b3-cb6231073aad'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.472963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5804256bf54a34987a945e1a5fd462', 'request_id': 'req-e26d23d9-c581-42f8-b5b3-cb6231073aad'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.488076 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] The token cache is being invalidate because project 3e5804256bf54a34987a945e1a5fd462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:33.488416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.489093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.496422 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1305/2629] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:33 2022] DELETE /identity/v3/projects/3e5804256bf54a34987a945e1a5fd462 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:33.497466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:33.509573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9a471-e336-430f-a322-f250bf5e3232 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:33.579867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:33.580614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.581081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.584700 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] PATH_INFO: `/v3/users/a1fb1f37d6c64c57b00b723620264432` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.591217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fb1f37d6c64c57b00b723620264432)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.593582 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.626712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:33.629689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.635442 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:33.659420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d23d9-c581-42f8-b5b3-cb6231073aad admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:33.669066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:33.670640 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.671084 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.671539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] PATH_INFO: `/v3/projects/b9fef62fd5894258b1afc5a360472400` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.673100 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fef62fd5894258b1afc5a360472400)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.676285 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.691677 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fb1f37d6c64c57b00b723620264432'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.708965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fb1f37d6c64c57b00b723620264432'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.728195 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fb1f37d6c64c57b00b723620264432'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.729731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fb1f37d6c64c57b00b723620264432'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.743367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f4ee2e31-cfd4-486a-8b7f-59a8aa2b0e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fb1f37d6c64c57b00b723620264432'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.751855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1325/2630] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:33 2022] DELETE /identity/v3/users/a1fb1f37d6c64c57b00b723620264432 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:33.763406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fef62fd5894258b1afc5a360472400', 'request_id': 'req-990bb2ff-05ad-4c45-a419-8a12daf2b72c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.771250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fef62fd5894258b1afc5a360472400', 'request_id': 'req-990bb2ff-05ad-4c45-a419-8a12daf2b72c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.782317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fef62fd5894258b1afc5a360472400', 'request_id': 'req-990bb2ff-05ad-4c45-a419-8a12daf2b72c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.801394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] The token cache is being invalidate because project b9fef62fd5894258b1afc5a360472400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:33.801738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.802435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-990bb2ff-05ad-4c45-a419-8a12daf2b72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.804525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1306/2631] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:33 2022] DELETE /identity/v3/projects/b9fef62fd5894258b1afc5a360472400 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:33.810390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.811096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.811573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.843687 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:33.845469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.940941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1326/2632] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:33.942379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.943377 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.943753 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.982361 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:33.984031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.993468 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1307/2633] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:33.998343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.014029 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f372298-2b8d-4eeb-85ef-fb737ff71a76 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.019014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.019756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.020037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.020306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.023601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:34.023913 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.025843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.040038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1327/2634] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:34.044872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.056567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7892c-8a73-44e2-8fd3-ba6b0b3023a8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.062791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.063690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.064214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.064670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.065496 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.068482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.102238 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1308/2635] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.107385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.121511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070c38-5b6c-4b93-99b0-3317ab0969dd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.128899 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.130977 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.211804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.213099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.213607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.214172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.215124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.217535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.220416 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:34.245537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1328/2636] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.250273 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.256669 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c6bcd-508c-4341-a557-831dc2ef2479 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.262279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b66a7-e9dc-4f4f-a05f-1a9027aa6517 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.263769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.264752 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.265375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.265805 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] PATH_INFO: `/v3/projects/f274225dc2854f8aa648c8aa1bcfb5b3` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.267276 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f274225dc2854f8aa648c8aa1bcfb5b3)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.268796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.269404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.269682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.269918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.270033 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.270624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.272208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.288928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1329/2637] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:34.293354 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.306414 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fd23b-89a2-4439-88a4-9d69a35d8f3d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.316189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.317217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.317680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.318098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.318943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.320627 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.325014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:34.333576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f274225dc2854f8aa648c8aa1bcfb5b3', 'request_id': 'req-369f8b86-73ed-4dbc-b807-fd74101aaf9d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.337909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1330/2638] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:34.344456 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f274225dc2854f8aa648c8aa1bcfb5b3', 'request_id': 'req-369f8b86-73ed-4dbc-b807-fd74101aaf9d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.345082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.353260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f274225dc2854f8aa648c8aa1bcfb5b3', 'request_id': 'req-369f8b86-73ed-4dbc-b807-fd74101aaf9d'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.362043 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d0dd4-874b-43ef-bb7c-3d562c2f51b8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.369458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] The token cache is being invalidate because project f274225dc2854f8aa648c8aa1bcfb5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:34.369874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.371161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.376325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1309/2639] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:34 2022] DELETE /identity/v3/projects/f274225dc2854f8aa648c8aa1bcfb5b3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:34.377297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.378775 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.441887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa1175-e333-4008-9b46-b61ee8aea7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.442523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa1175-e333-4008-9b46-b61ee8aea7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.442821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa1175-e333-4008-9b46-b61ee8aea7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.443118 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa1175-e333-4008-9b46-b61ee8aea7ab admin admin] PATH_INFO: `/v3/projects/81ebddce0aa6479dab57c7c239d7b848/users/114267b7c3794c3b885daa47e022ef30/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.469812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa1175-e333-4008-9b46-b61ee8aea7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ebddce0aa6479dab57c7c239d7b848, user_id=114267b7c3794c3b885daa47e022ef30, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.472797 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa1175-e333-4008-9b46-b61ee8aea7ab admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.479885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f8b86-73ed-4dbc-b807-fd74101aaf9d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.486430 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.487118 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.487387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.487711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] PATH_INFO: `/v3/users/560c548ce3b142948bd273fa0e34b1dc` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.489160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=560c548ce3b142948bd273fa0e34b1dc)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.491317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.503864 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1331/2640] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:34 2022] PUT /identity/v3/projects/81ebddce0aa6479dab57c7c239d7b848/users/114267b7c3794c3b885daa47e022ef30/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:34.506798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.507082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.507355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.542856 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] MFA Rules not processed for user `114267b7c3794c3b885daa47e022ef30`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:34.544124 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.622145 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560c548ce3b142948bd273fa0e34b1dc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.631381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560c548ce3b142948bd273fa0e34b1dc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.631954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1332/2641] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:34.632205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560c548ce3b142948bd273fa0e34b1dc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.637530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.639359 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.642670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560c548ce3b142948bd273fa0e34b1dc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.649373 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:34.664726 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560c548ce3b142948bd273fa0e34b1dc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:34.673887 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1310/2642] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:34 2022] DELETE /identity/v3/users/560c548ce3b142948bd273fa0e34b1dc => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:34.725743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd7ef9-b3f5-4e59-a5e6-d615261cb6ac None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.742327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:34.807955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.808844 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.809362 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.809745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.810933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.813615 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.853833 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1333/2643] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.855633 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.857489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.874806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95559c0a-801f-4557-b8e1-a6ef9abf83e1 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.986016 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.986925 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.987477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.987975 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.988882 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.991420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.028546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1311/2644] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:35.029392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.045453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc6f1c-b5d7-4a34-afad-70ccb813a6b6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.052915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.053780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.054218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.054655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.056039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.059578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.082662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1334/2645] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:35.084267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.096205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb5e9e-84fd-4254-9d96-5a95e401c0ea admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.097330 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.099235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.101603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.102167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.102451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.102721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.103364 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.105649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.120523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938e6f3-fabe-4c1b-ba68-2754c244c87d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.121705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1312/2646] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:35.125937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.129760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.130774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.130903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.131505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] PATH_INFO: `/v3/projects/09049ee1f9d141198d6e3d5669850df8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.134165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=09049ee1f9d141198d6e3d5669850df8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.137329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.140421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16093f57-01e6-4cf8-9274-a1d5124aea22 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.146303 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbeb071-b4bf-4128-bc1c-a52febdb4d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.146962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbeb071-b4bf-4128-bc1c-a52febdb4d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.147304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbeb071-b4bf-4128-bc1c-a52febdb4d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.147541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbeb071-b4bf-4128-bc1c-a52febdb4d3b admin admin] PATH_INFO: `/v3/projects/0bc1d0b8f79e44739d1ebd75b663379b/users/f5576bf71a5742609563ea0ac997759f/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.172943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbeb071-b4bf-4128-bc1c-a52febdb4d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc1d0b8f79e44739d1ebd75b663379b, user_id=f5576bf71a5742609563ea0ac997759f, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.176403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbeb071-b4bf-4128-bc1c-a52febdb4d3b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.205520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1313/2647] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:35 2022] PUT /identity/v3/projects/0bc1d0b8f79e44739d1ebd75b663379b/users/f5576bf71a5742609563ea0ac997759f/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:35.207487 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.208002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.208571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.218648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09049ee1f9d141198d6e3d5669850df8', 'request_id': 'req-131304ba-a134-4ad2-9a49-54dad95ee3bc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:35.232542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09049ee1f9d141198d6e3d5669850df8', 'request_id': 'req-131304ba-a134-4ad2-9a49-54dad95ee3bc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:35.242845 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09049ee1f9d141198d6e3d5669850df8', 'request_id': 'req-131304ba-a134-4ad2-9a49-54dad95ee3bc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:35.249701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] MFA Rules not processed for user `f5576bf71a5742609563ea0ac997759f`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:35.251104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.258763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] The token cache is being invalidate because project 09049ee1f9d141198d6e3d5669850df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:35.259061 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:35.259939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-131304ba-a134-4ad2-9a49-54dad95ee3bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:35.266949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1335/2648] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:35 2022] DELETE /identity/v3/projects/09049ee1f9d141198d6e3d5669850df8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:35.268625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93020eec-256b-4ed0-9c49-e75f4702d681 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.268965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93020eec-256b-4ed0-9c49-e75f4702d681 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.269259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93020eec-256b-4ed0-9c49-e75f4702d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.305317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-93020eec-256b-4ed0-9c49-e75f4702d681 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:35.307523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93020eec-256b-4ed0-9c49-e75f4702d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.333272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1314/2649] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:35.336305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.337653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.378683 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:35.405100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1336/2650] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:35.413688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.413993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.414649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.438454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03e728-fa6e-43df-aad0-214b91f47d30 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.447717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:35.447951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.448536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.448857 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.449090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.449217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.449758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.451798 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.458262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1337/2651] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:35.462690 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.476455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbc76-7c39-4c10-9b1f-31ad13071a78 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.483980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.484608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.484992 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.485338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.486059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:35.486427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.488156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.489945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1315/2652] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:35.493788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.496127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:35.515505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1338/2653] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:35.518499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.531814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0f3eb-4df7-4936-b6b3-fcd25971f241 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.579537 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bb9a0-81d7-452c-8fdf-bdbdc9f768e8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.585794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.586552 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.586878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.587382 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.588651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.593960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.611426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.612174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.612451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.612731 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.613379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.615467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.622259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1316/2654] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:35.623883 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.639237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a4a0-cbe4-4ba5-9c5c-35dd3ebf4fc7 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.645719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.646485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.646837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.649947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.649947 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.652929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.660458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1339/2655] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:35.664716 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.671109 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1317/2656] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:35.672291 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.687863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5f1cd-2dbf-4e07-8000-aac5556625d8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.742472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004545b-26ca-4abc-97d3-9e83937e42dc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.747922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.748744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.749228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.749511 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.750122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.752066 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.771511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.772355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.772694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.773045 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] PATH_INFO: `/v3/projects/8ee820b898d846a8ad6f159a53ccc015/users/98121d0c3cf041019fa7c79bb5c5a8aa/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.784946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1340/2657] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:35.788396 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.796124 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee820b898d846a8ad6f159a53ccc015, user_id=98121d0c3cf041019fa7c79bb5c5a8aa, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.799259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.801271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6deb6-4435-4b00-abaa-52481de220fb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.809021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.810281 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.810735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.811313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.812074 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.814121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.815972 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1318/2658] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:35 2022] PUT /identity/v3/projects/8ee820b898d846a8ad6f159a53ccc015/users/98121d0c3cf041019fa7c79bb5c5a8aa/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:35.820018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.828746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1341/2659] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:35.832290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.846575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27786f43-e1c8-4370-bfc0-0774e49fa8c8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.849692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:35.939670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ed4a8-3988-4511-8274-76e012f66643 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.944777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.945507 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.946133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.946499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.947245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.949847 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.952058 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.954081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.954563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.955659 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.956507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.960154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.963279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:35.967745 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1319/2660] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:35.968812 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.987128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1342/2661] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:35.990818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.991751 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2de86c-4442-4118-9c07-37e78d6e3337 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.001501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6822-4a8e-4450-bc81-f3d0e9575646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.002247 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6822-4a8e-4450-bc81-f3d0e9575646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.002654 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6822-4a8e-4450-bc81-f3d0e9575646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.004058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6822-4a8e-4450-bc81-f3d0e9575646 admin admin] PATH_INFO: `/v3/projects/8ee820b898d846a8ad6f159a53ccc015/users/98121d0c3cf041019fa7c79bb5c5a8aa/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.010884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1bfe1-20b1-404a-906c-65463c75c5b4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.012929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6822-4a8e-4450-bc81-f3d0e9575646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee820b898d846a8ad6f159a53ccc015, user_id=98121d0c3cf041019fa7c79bb5c5a8aa, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.015880 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6822-4a8e-4450-bc81-f3d0e9575646 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.016316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a9fe2-8cce-4c7c-a7cf-ca3341506a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.017222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a9fe2-8cce-4c7c-a7cf-ca3341506a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.017573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a9fe2-8cce-4c7c-a7cf-ca3341506a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.017938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a9fe2-8cce-4c7c-a7cf-ca3341506a18 admin admin] PATH_INFO: `/v3/projects/599736fbc7fb475b9ce98e4cca088b32/users/ba04e74f5b0447b5952b63089406dfec/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.037583 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a9fe2-8cce-4c7c-a7cf-ca3341506a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599736fbc7fb475b9ce98e4cca088b32, user_id=ba04e74f5b0447b5952b63089406dfec, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.041056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a9fe2-8cce-4c7c-a7cf-ca3341506a18 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.058516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1320/2662] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:35 2022] PUT /identity/v3/projects/8ee820b898d846a8ad6f159a53ccc015/users/98121d0c3cf041019fa7c79bb5c5a8aa/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.089460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1343/2663] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:35 2022] PUT /identity/v3/projects/599736fbc7fb475b9ce98e4cca088b32/users/ba04e74f5b0447b5952b63089406dfec/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.125675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.126119 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.126503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.153637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.154761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.154761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.163942 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] MFA Rules not processed for user `98121d0c3cf041019fa7c79bb5c5a8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:36.167189 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:36.189453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] MFA Rules not processed for user `ba04e74f5b0447b5952b63089406dfec`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:36.190735 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:36.270445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1321/2664] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:36.280970 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.285477 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:36.295002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1344/2665] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:36.296374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.365603 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51639e98-4047-4b2b-9094-5db7d6be7324 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.372937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.372937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.372937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.372937 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.376480 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:36.386341 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d23c1-4208-40ba-a5ae-f37c561e485c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.391684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.392351 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.392707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.393070 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.396022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.396022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.429534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:36.432800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1345/2666] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:36 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:36.437466 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.463682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.467620 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.483019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1322/2667] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:36.505521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:36.524707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7747-3b5d-4be5-8d32-d77e0f6d1d8b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.530978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.531639 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.531920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.532191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.532846 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.534803 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.563164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1346/2668] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:36 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:36.567178 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.587005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8538d-cf45-442a-b7bd-26aaed4d68a4 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.597136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.597754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.598031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.598304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.598953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.600582 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.613617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1323/2669] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:36.617895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.635180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbdde2-43f8-430e-973f-84bafb1337a9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.643879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.644907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.645546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.646015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] PATH_INFO: `/v3/projects/8684d67c79434d11849dcb337f3668df/users/0d100f1ebc0348cbba1d6e3350c8ae1b/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.680188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8684d67c79434d11849dcb337f3668df, user_id=0d100f1ebc0348cbba1d6e3350c8ae1b, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.683100 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.705950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1347/2670] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:36 2022] PUT /identity/v3/projects/8684d67c79434d11849dcb337f3668df/users/0d100f1ebc0348cbba1d6e3350c8ae1b/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.710893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.727639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a2d4-044b-48a6-af0d-a8becf4d6b40 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.830649 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2df85e-aae1-443a-b23c-678594fa6720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.831284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2df85e-aae1-443a-b23c-678594fa6720 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.831565 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2df85e-aae1-443a-b23c-678594fa6720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.831834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2df85e-aae1-443a-b23c-678594fa6720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.832483 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2df85e-aae1-443a-b23c-678594fa6720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.834213 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2df85e-aae1-443a-b23c-678594fa6720 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.856399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1324/2671] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:36.857089 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.872381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06eff9b-e938-46b7-8cd6-b28d44a07ebe admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.895664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.896590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.897585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.898037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] PATH_INFO: `/v3/projects/8684d67c79434d11849dcb337f3668df/users/0d100f1ebc0348cbba1d6e3350c8ae1b/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.900980 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8684d67c79434d11849dcb337f3668df, user_id=0d100f1ebc0348cbba1d6e3350c8ae1b, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.904819 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.921214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1348/2672] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:36 2022] PUT /identity/v3/projects/8684d67c79434d11849dcb337f3668df/users/0d100f1ebc0348cbba1d6e3350c8ae1b/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.935323 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.935595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.936005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.977316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] MFA Rules not processed for user `0d100f1ebc0348cbba1d6e3350c8ae1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:36.977316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:37.079331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1325/2673] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:37.092914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.162920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ed143-7862-436b-bf9c-4d8031ddc265 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.171024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.171751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.172078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.172429 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.184733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.186508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.205568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1349/2674] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:37.305966 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.326018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaadb19-9a27-4785-b440-8b6d77c3f161 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3fb9a-d001-4856-971e-592bc042ae6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fb9a-d001-4856-971e-592bc042ae6d service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fb9a-d001-4856-971e-592bc042ae6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.348982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fb9a-d001-4856-971e-592bc042ae6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.403056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:37.416712 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3fb9a-d001-4856-971e-592bc042ae6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.418457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3fb9a-d001-4856-971e-592bc042ae6d service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.429905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1326/2675] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:37.755880 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.835696 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6d495-5ca0-4b67-b953-785ff21db2fd service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.841728 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.842376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.842650 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.842920 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] PATH_INFO: `/v3/users/ba04e74f5b0447b5952b63089406dfec` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.844093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba04e74f5b0447b5952b63089406dfec)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.846084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.906553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:37.916112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba04e74f5b0447b5952b63089406dfec'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:37.924605 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba04e74f5b0447b5952b63089406dfec'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:37.938868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba04e74f5b0447b5952b63089406dfec'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:37.939620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba04e74f5b0447b5952b63089406dfec'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:37.948614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba04e74f5b0447b5952b63089406dfec'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:37.955444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1350/2676] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:37 2022] DELETE /identity/v3/users/ba04e74f5b0447b5952b63089406dfec => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:37.960293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.960578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.960962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.984250 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:37.985626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:38.094828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1327/2677] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:38.107383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.108999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:38.187385 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afe785-b3d7-4468-8e51-d5bf3218098d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.197973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.198541 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.198973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.199425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.210227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.211970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.236648 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1351/2678] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:38.248242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.249764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:38.320545 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc2952-8c40-4870-b3c6-1ed291aa03d8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.326339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.327089 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.327511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.328021 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.330692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:38.478710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.480574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.500671 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1328/2679] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:38.702292 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.704506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:38.733724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9bff2-6e98-4786-87e4-6f5df9e7868a service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.756098 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.756932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.757406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.757780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] PATH_INFO: `/v3/projects/599736fbc7fb475b9ce98e4cca088b32` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.759155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=599736fbc7fb475b9ce98e4cca088b32)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.762503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.830156 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599736fbc7fb475b9ce98e4cca088b32', 'request_id': 'req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:38.840015 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599736fbc7fb475b9ce98e4cca088b32', 'request_id': 'req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:38.850192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599736fbc7fb475b9ce98e4cca088b32', 'request_id': 'req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:38.864743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] The token cache is being invalidate because project 599736fbc7fb475b9ce98e4cca088b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:38.864994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:38.865777 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:38.868406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1352/2680] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:38 2022] DELETE /identity/v3/projects/599736fbc7fb475b9ce98e4cca088b32 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:38.872960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.910714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae27b7-7afb-40a8-8fc2-00d983f58f6b service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.989383 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:39.015826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:39.016618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.016978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.017255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] PATH_INFO: `/v3/users/0d100f1ebc0348cbba1d6e3350c8ae1b` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.018402 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d100f1ebc0348cbba1d6e3350c8ae1b)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:39.020421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:39.100743 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d100f1ebc0348cbba1d6e3350c8ae1b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.113005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d100f1ebc0348cbba1d6e3350c8ae1b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.114470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d100f1ebc0348cbba1d6e3350c8ae1b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.123314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d100f1ebc0348cbba1d6e3350c8ae1b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.138894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d100f1ebc0348cbba1d6e3350c8ae1b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.145720 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1329/2681] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:38 2022] DELETE /identity/v3/users/0d100f1ebc0348cbba1d6e3350c8ae1b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:39.293007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:39.294444 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:39.386125 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:39.396251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f6aab-ac3c-44b4-928a-ec4b130128dc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:39.482230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:39.483107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.483107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.483401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.485449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:39.516142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:39.518111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:39.569717 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:39.571401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c158bf-f175-4562-8f03-e09942bd6804 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:39.590114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1353/2682] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:39.631560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760dd1a-4d6d-4bf1-b9c7-7ed7491d550d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:39.640271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:39.641039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.641387 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.641721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] PATH_INFO: `/v3/projects/8684d67c79434d11849dcb337f3668df` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.643271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8684d67c79434d11849dcb337f3668df)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:39.646141 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:39.732440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8684d67c79434d11849dcb337f3668df', 'request_id': 'req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.757536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8684d67c79434d11849dcb337f3668df', 'request_id': 'req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.764676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8684d67c79434d11849dcb337f3668df', 'request_id': 'req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.772161 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:39.795825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] The token cache is being invalidate because project 8684d67c79434d11849dcb337f3668df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:39.796116 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.798473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-23187ba7-5a9e-47ea-ad5f-76b5e22f09cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:39.804077 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1330/2683] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:39 2022] DELETE /identity/v3/projects/8684d67c79434d11849dcb337f3668df => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:39.809899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.810389 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.810470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.837712 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:39.844986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:39.935963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1354/2684] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:39.945869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.946172 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.946420 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.981618 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:39.982946 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:39.994885 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1331/2685] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:39.996217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.006889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ea324-7d23-47ae-b51a-56e601eac027 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.014472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.015439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.015989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.016586 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.017783 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:40.018484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.020585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.039467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1355/2686] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:40.044234 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.054121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb9bf2-2e3a-4c78-a369-738d412bd311 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.059985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.060730 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.061125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.061441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.062173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.064207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.101498 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1332/2687] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:40 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:40.105949 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.116895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1905-5815-4630-8d6f-2562b8783857 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.158895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:40.181374 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.182710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.182987 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.183255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.183889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.185865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.209046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1356/2688] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:40 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:40.213246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.225867 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719a1c4-0c33-46aa-9df3-ac75764dfc5a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.231529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.232342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.232806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.233299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.234102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.235744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.247666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1333/2689] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:40.251512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.268230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b6217-0cbc-4061-b298-225776e5e4c9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.273569 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544351a0-494f-4a6b-a52e-6ca5d4f0a8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.274168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544351a0-494f-4a6b-a52e-6ca5d4f0a8eb admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.274449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544351a0-494f-4a6b-a52e-6ca5d4f0a8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.274724 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544351a0-494f-4a6b-a52e-6ca5d4f0a8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.276521 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544351a0-494f-4a6b-a52e-6ca5d4f0a8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.281455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544351a0-494f-4a6b-a52e-6ca5d4f0a8eb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.293606 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1357/2690] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:40.298028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.309797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cfc7b-8033-4705-9e6c-ebc640f1c60b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.314541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.315193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.315485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.315862 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] PATH_INFO: `/v3/projects/d5e2d5c2e0ce43f9b352daee2f2bf5f9/users/467f45f0cfdb4ef58f893e633075e3ae/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.332605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e2d5c2e0ce43f9b352daee2f2bf5f9, user_id=467f45f0cfdb4ef58f893e633075e3ae, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.335353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.351993 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1334/2691] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:40 2022] PUT /identity/v3/projects/d5e2d5c2e0ce43f9b352daee2f2bf5f9/users/467f45f0cfdb4ef58f893e633075e3ae/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:40.362814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f4b8-46fd-461b-93df-58703e108d7e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.363551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f4b8-46fd-461b-93df-58703e108d7e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.363886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f4b8-46fd-461b-93df-58703e108d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.399470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5bb8f4b8-46fd-461b-93df-58703e108d7e None None] MFA Rules not processed for user `467f45f0cfdb4ef58f893e633075e3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:40.401610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5bb8f4b8-46fd-461b-93df-58703e108d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:40.482603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1358/2692] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:40.495448 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.497441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:40.562642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d7df-7676-451d-84c1-6148d3951c9f admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.569271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.569973 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.570318 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.570777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.581136 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.582714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.595295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1335/2693] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:42.148186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14125983-a52d-419d-807c-8ddbfe482afc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:42.148736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14125983-a52d-419d-807c-8ddbfe482afc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:42.148736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14125983-a52d-419d-807c-8ddbfe482afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:42.176235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-14125983-a52d-419d-807c-8ddbfe482afc None None] MFA Rules not processed for user `467f45f0cfdb4ef58f893e633075e3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:42.177514 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-14125983-a52d-419d-807c-8ddbfe482afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:42.187637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1359/2694] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:43.801900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:43.803423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.875944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091de72-fb8a-4709-9e54-4c4983523e92 service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:43.880491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:43.881139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.881431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.881678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] PATH_INFO: `/v3/users/114267b7c3794c3b885daa47e022ef30` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.882799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114267b7c3794c3b885daa47e022ef30)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:43.891535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:43.936936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:43.947092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114267b7c3794c3b885daa47e022ef30'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.954871 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114267b7c3794c3b885daa47e022ef30'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.955595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114267b7c3794c3b885daa47e022ef30'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.963651 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114267b7c3794c3b885daa47e022ef30'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.976735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114267b7c3794c3b885daa47e022ef30'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.982954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1336/2695] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:43 2022] DELETE /identity/v3/users/114267b7c3794c3b885daa47e022ef30 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:43.987272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.987781 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.987895 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.006227 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:44.007409 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:44.067081 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1360/2696] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:44.077369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.078804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:44.136570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae28ee0-3873-4abf-873a-823a2e90c5f6 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.140179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.140795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.141079 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.141353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.150254 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.152661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.161929 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1337/2697] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:44.425182 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.426686 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:44.441278 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c477ba3-d755-4949-9292-74ed8065a6d4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.446037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.446671 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.446950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.447986 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] PATH_INFO: `/v3/projects/81ebddce0aa6479dab57c7c239d7b848` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.450045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ebddce0aa6479dab57c7c239d7b848)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.452007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.490603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:44.501942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ebddce0aa6479dab57c7c239d7b848', 'request_id': 'req-68b55911-9000-40ac-a060-5ebf8e83d378'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.508196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ebddce0aa6479dab57c7c239d7b848', 'request_id': 'req-68b55911-9000-40ac-a060-5ebf8e83d378'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.516111 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ebddce0aa6479dab57c7c239d7b848', 'request_id': 'req-68b55911-9000-40ac-a060-5ebf8e83d378'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.526518 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] The token cache is being invalidate because project 81ebddce0aa6479dab57c7c239d7b848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:44.526800 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.527426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.528941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1361/2698] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:44 2022] DELETE /identity/v3/projects/81ebddce0aa6479dab57c7c239d7b848 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:44.532943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.542677 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a07e20-f3ae-4a24-acc0-748a8fb48b1b service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.600146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.600829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.601107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.601376 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] PATH_INFO: `/v3/users/f5576bf71a5742609563ea0ac997759f` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.602512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5576bf71a5742609563ea0ac997759f)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.605614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.657912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5576bf71a5742609563ea0ac997759f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.663296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:44.665244 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5576bf71a5742609563ea0ac997759f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.667231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5576bf71a5742609563ea0ac997759f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.674940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5576bf71a5742609563ea0ac997759f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.687851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5576bf71a5742609563ea0ac997759f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:44.694233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1338/2699] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:44 2022] DELETE /identity/v3/users/f5576bf71a5742609563ea0ac997759f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:45.055452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:45.057439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:45.072793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b55911-9000-40ac-a060-5ebf8e83d378 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:45.132272 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:45.132973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:45.133254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:45.133546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] PATH_INFO: `/v3/projects/0bc1d0b8f79e44739d1ebd75b663379b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:45.134823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc1d0b8f79e44739d1ebd75b663379b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:45.137190 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:45.184681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc1d0b8f79e44739d1ebd75b663379b', 'request_id': 'req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.190888 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc1d0b8f79e44739d1ebd75b663379b', 'request_id': 'req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.197911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc1d0b8f79e44739d1ebd75b663379b', 'request_id': 'req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.208314 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] The token cache is being invalidate because project 0bc1d0b8f79e44739d1ebd75b663379b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:45.208599 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.209253 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.210709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1362/2700] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:45 2022] DELETE /identity/v3/projects/0bc1d0b8f79e44739d1ebd75b663379b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:45.215306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:45.217046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:45.230335 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03cb4a-c8d9-4b7a-a6f8-234c761898ed admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:45.303491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:45.304329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:45.304615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:45.304930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] PATH_INFO: `/v3/users/98121d0c3cf041019fa7c79bb5c5a8aa` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:45.306111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98121d0c3cf041019fa7c79bb5c5a8aa)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:45.308150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:45.497240 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98121d0c3cf041019fa7c79bb5c5a8aa'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.505046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98121d0c3cf041019fa7c79bb5c5a8aa'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.505788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98121d0c3cf041019fa7c79bb5c5a8aa'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.515167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98121d0c3cf041019fa7c79bb5c5a8aa'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.530260 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0095dd62-e60d-4aa2-9534-8624501d21f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98121d0c3cf041019fa7c79bb5c5a8aa'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:45.539852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1339/2701] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:45 2022] DELETE /identity/v3/users/98121d0c3cf041019fa7c79bb5c5a8aa => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:45.830594 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:45.832076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:45.847192 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9c8d6-2d9b-4754-bad0-f2a6d073a1e7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:45.906338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:45.914858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:45.915652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:45.916116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:45.916611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] PATH_INFO: `/v3/projects/8ee820b898d846a8ad6f159a53ccc015` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:45.918050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee820b898d846a8ad6f159a53ccc015)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:45.920023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:45.997321 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee820b898d846a8ad6f159a53ccc015', 'request_id': 'req-9d04fef6-88b3-471e-a41e-11ce07d1f10f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:46.005629 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee820b898d846a8ad6f159a53ccc015', 'request_id': 'req-9d04fef6-88b3-471e-a41e-11ce07d1f10f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:46.015308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee820b898d846a8ad6f159a53ccc015', 'request_id': 'req-9d04fef6-88b3-471e-a41e-11ce07d1f10f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:46.027891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] The token cache is being invalidate because project 8ee820b898d846a8ad6f159a53ccc015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:46.028438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:46.029153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9d04fef6-88b3-471e-a41e-11ce07d1f10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:46.030770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1363/2702] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:45 2022] DELETE /identity/v3/projects/8ee820b898d846a8ad6f159a53ccc015 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:46.035648 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.035930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.036197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.065511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:46.069004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:46.143174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:46.177632 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.178176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.178176 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.179488 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1340/2703] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:46.200415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:46.201973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:46.217889 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1364/2704] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:46.221843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.232988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954bc34-06de-420a-baf0-9d69684fbeeb None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.239103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.239807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.240170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.240510 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.241426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:46.241796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.243541 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.259327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1341/2705] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:46.264221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.279284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315b76-16fa-4a7f-ac05-201a558142f4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.297477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.297477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.297477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.297477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.298151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.298151 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.332834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1365/2706] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:46 2022] POST /identity/v3/projects => generated 397 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:46.334653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.350450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9a37d-98f1-4c58-a3dc-09ea959815d9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.434672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.435214 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.435656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.435965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.437037 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.440571 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.468863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1342/2707] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:46 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:46.473375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.485825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58258-d0ee-49cf-98f4-665aa79d79f7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.491628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.492400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.492709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.493082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.493747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.495424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.506758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1366/2708] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:46.511573 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.523955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89314750-74fb-4203-be40-f3a91160ed6a admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.529742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013a392-6459-497e-a296-dcaec08349f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.530474 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013a392-6459-497e-a296-dcaec08349f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.530764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013a392-6459-497e-a296-dcaec08349f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.531485 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013a392-6459-497e-a296-dcaec08349f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.531670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013a392-6459-497e-a296-dcaec08349f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.533436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013a392-6459-497e-a296-dcaec08349f8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.562220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1343/2709] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:46.566613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.578863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002b7e-9115-41c1-9dae-0256ef162120 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.584720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.585427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.585646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.585912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] PATH_INFO: `/v3/projects/34d5aed8849b4714909dd4ed7eb38ac7/users/4c1053e487d245afa41b19a90e240bde/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.606401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d5aed8849b4714909dd4ed7eb38ac7, user_id=4c1053e487d245afa41b19a90e240bde, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.614681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.630742 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1367/2710] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:46 2022] PUT /identity/v3/projects/34d5aed8849b4714909dd4ed7eb38ac7/users/4c1053e487d245afa41b19a90e240bde/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:46.643208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.643484 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.643747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.678851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] MFA Rules not processed for user `4c1053e487d245afa41b19a90e240bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:46.680096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:46.798019 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1344/2711] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:46.810645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.812894 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:46.888218 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6897ee8-a888-4752-b1ce-aa76859819b8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:46.898474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75827c1e-9808-451b-a921-fd720eac93c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:46.899119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827c1e-9808-451b-a921-fd720eac93c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.899402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827c1e-9808-451b-a921-fd720eac93c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.899652 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827c1e-9808-451b-a921-fd720eac93c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.915646 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827c1e-9808-451b-a921-fd720eac93c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:46.917684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827c1e-9808-451b-a921-fd720eac93c9 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:46.928308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1368/2712] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:47.209907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:47.211558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:47.287785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:47.299123 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793847a-da66-41a5-926f-9e4914319be9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:47.308130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:47.308756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.309056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.309328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.314201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:47.330087 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:47.331951 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194252e-ddb7-4ff3-a36f-25f4180ca3bd service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:47.343173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1345/2713] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:47.586903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fedfae-c4d1-4251-a398-447033875fb9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.587393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fedfae-c4d1-4251-a398-447033875fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.587489 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fedfae-c4d1-4251-a398-447033875fb9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.589727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1369/2714] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:47.596407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49e42b-6669-4aa0-ba34-ca0e88061e4c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.596690 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49e42b-6669-4aa0-ba34-ca0e88061e4c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.597012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49e42b-6669-4aa0-ba34-ca0e88061e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.626727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-fe49e42b-6669-4aa0-ba34-ca0e88061e4c None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:47.628562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fe49e42b-6669-4aa0-ba34-ca0e88061e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:47.714458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1346/2715] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:36:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:47.726197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fedfae-c4d1-4251-a398-447033875fb9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:47.727926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e3fedfae-c4d1-4251-a398-447033875fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:47.764859 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:47.797858 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fedfae-c4d1-4251-a398-447033875fb9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:47.801571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9a37e-2d24-49a2-8a24-6b19ec52e4ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4bd5bc35c41458b9ba20d87738cde16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:47.802337 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9a37e-2d24-49a2-8a24-6b19ec52e4ed service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.802683 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9a37e-2d24-49a2-8a24-6b19ec52e4ed service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.803018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9a37e-2d24-49a2-8a24-6b19ec52e4ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.814792 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9a37e-2d24-49a2-8a24-6b19ec52e4ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:47.816235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9a37e-2d24-49a2-8a24-6b19ec52e4ed service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:47.827656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1370/2716] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:47.912044 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04262b7-b635-447e-8f12-ad0aadfd1fe1 None None] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.912186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04262b7-b635-447e-8f12-ad0aadfd1fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.912458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04262b7-b635-447e-8f12-ad0aadfd1fe1 None None] PATH_INFO: `/` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.914142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1347/2717] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:47.920638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38f24d-bb94-4861-92ce-0fcb25cbb3f5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.921064 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38f24d-bb94-4861-92ce-0fcb25cbb3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.921435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38f24d-bb94-4861-92ce-0fcb25cbb3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.942347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3c38f24d-bb94-4861-92ce-0fcb25cbb3f5 None None] MFA Rules not processed for user `12a2363821f445db980712d3e491e791`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:47.943544 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3c38f24d-bb94-4861-92ce-0fcb25cbb3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:47.950211 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1371/2718] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:36:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:47.961921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04262b7-b635-447e-8f12-ad0aadfd1fe1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:47.975122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04262b7-b635-447e-8f12-ad0aadfd1fe1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:47.979670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b472c-b307-4044-b067-2b23bd45e30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:47.980261 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b472c-b307-4044-b067-2b23bd45e30a service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.980533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b472c-b307-4044-b067-2b23bd45e30a service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.980983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b472c-b307-4044-b067-2b23bd45e30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.988424 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b472c-b307-4044-b067-2b23bd45e30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:47.989914 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b472c-b307-4044-b067-2b23bd45e30a service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:47.998330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1348/2719] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:48.084379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae03bd-f2d0-4591-b1be-d63bfb98e2b2 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.084661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae03bd-f2d0-4591-b1be-d63bfb98e2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.084967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae03bd-f2d0-4591-b1be-d63bfb98e2b2 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.086343 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1372/2720] 10.222.0.81 () {54 vars in 921 bytes} [Wed Aug 24 03:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:48.091827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085380-7c61-49fb-a34f-b04afea97d03 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.092112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085380-7c61-49fb-a34f-b04afea97d03 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.092386 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085380-7c61-49fb-a34f-b04afea97d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.111980 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-87085380-7c61-49fb-a34f-b04afea97d03 None None] MFA Rules not processed for user `b8afb31c267946b7851cb1c22b62b6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:48.113202 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-87085380-7c61-49fb-a34f-b04afea97d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:48.123324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1349/2721] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Aug 24 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:48.133761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae03bd-f2d0-4591-b1be-d63bfb98e2b2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:48.145915 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae03bd-f2d0-4591-b1be-d63bfb98e2b2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:48.149498 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:48.150076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.151014 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.151397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.161524 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:48.162943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:48.174159 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1373/2722] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:52.840754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.841630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.841630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.861142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:52.862347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.919727 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1350/2723] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:52.932147 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.933601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.001921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce68c-0f1f-4f6f-97be-93831c979792 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.006855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83c160-3b67-4884-9b9f-8fc75f5046b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.007454 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83c160-3b67-4884-9b9f-8fc75f5046b6 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.007762 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83c160-3b67-4884-9b9f-8fc75f5046b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.008037 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83c160-3b67-4884-9b9f-8fc75f5046b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.016713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83c160-3b67-4884-9b9f-8fc75f5046b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.018369 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83c160-3b67-4884-9b9f-8fc75f5046b6 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.029045 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1374/2724] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:36:54.195380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:54.277432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997e137-e2a8-4091-b963-c299ba7d857e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:54.284299 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:54.285536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.285930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.286482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] PATH_INFO: `/v3/users/4c1053e487d245afa41b19a90e240bde` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.288223 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1053e487d245afa41b19a90e240bde)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:54.291511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:54.298960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=100 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:54.358293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1053e487d245afa41b19a90e240bde'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:54.366625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1053e487d245afa41b19a90e240bde'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:54.367515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1053e487d245afa41b19a90e240bde'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:54.378663 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1053e487d245afa41b19a90e240bde'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:54.395526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1053e487d245afa41b19a90e240bde'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:54.403162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1351/2725] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:54 2022] DELETE /identity/v3/users/4c1053e487d245afa41b19a90e240bde => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:54.408059 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.408542 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.408676 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.431114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:54.433036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:54.548306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1375/2726] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:54.561197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:54.563522 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:54.644722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827264a-40be-4562-a2c4-503741febb5c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:54.648458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19bda-b7b8-4d18-8fad-48a1c464f5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:54.649163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19bda-b7b8-4d18-8fad-48a1c464f5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.649503 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19bda-b7b8-4d18-8fad-48a1c464f5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.649777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19bda-b7b8-4d18-8fad-48a1c464f5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.659251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19bda-b7b8-4d18-8fad-48a1c464f5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:54.660666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19bda-b7b8-4d18-8fad-48a1c464f5b7 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:54.670052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1352/2727] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:55.080234 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:55.082191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:55.099899 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df168d8-430e-42c9-8e75-1a75df4587c0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:55.106688 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:55.107919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.107919 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.108141 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] PATH_INFO: `/v3/projects/34d5aed8849b4714909dd4ed7eb38ac7` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.109505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d5aed8849b4714909dd4ed7eb38ac7)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:55.111552 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:55.157961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:55.176026 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d5aed8849b4714909dd4ed7eb38ac7', 'request_id': 'req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:55.183458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d5aed8849b4714909dd4ed7eb38ac7', 'request_id': 'req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:55.196702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d5aed8849b4714909dd4ed7eb38ac7', 'request_id': 'req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:55.209172 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] The token cache is being invalidate because project 34d5aed8849b4714909dd4ed7eb38ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:55.209456 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:55.210140 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:55.211620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1376/2728] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:55 2022] DELETE /identity/v3/projects/34d5aed8849b4714909dd4ed7eb38ac7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:04.665715 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:05.206121 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:09.505562 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdadd6-83c5-498e-8acc-8496b8bb9c2a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:09.506041 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdadd6-83c5-498e-8acc-8496b8bb9c2a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:09.506134 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdadd6-83c5-498e-8acc-8496b8bb9c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:09.528916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-25bdadd6-83c5-498e-8acc-8496b8bb9c2a None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:09.530162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-25bdadd6-83c5-498e-8acc-8496b8bb9c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:09.585685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1353/2729] 10.222.0.81 () {58 vars in 967 bytes} [Wed Aug 24 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:09.598625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:09.600039 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:09.671324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd6bb0-7b7e-4e13-87f8-a3eabf9d8a9b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:09.676329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:09.676961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:09.677240 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:09.677513 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:09.686163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:09.687744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:09.695720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1377/2730] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:10.449137 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4a3dc-1f22-4c34-913e-74983b2307fc None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:10.449499 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4a3dc-1f22-4c34-913e-74983b2307fc None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:10.449870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4a3dc-1f22-4c34-913e-74983b2307fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:10.469828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a3e4a3dc-1f22-4c34-913e-74983b2307fc None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:10.471049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a3e4a3dc-1f22-4c34-913e-74983b2307fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:10.478301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1354/2731] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:37:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:10.490925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:10.501961 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375abee-900e-4fdf-9f31-9618f4ee5be4 service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:10.507004 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:10.508008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:10.508008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:10.508224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:10.517508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:10.519107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:10.528217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1378/2732] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:11.071673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5b735-428e-4f85-9cdb-891b7ac6a857 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.072140 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5b735-428e-4f85-9cdb-891b7ac6a857 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.073355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5b735-428e-4f85-9cdb-891b7ac6a857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.092920 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8ef5b735-428e-4f85-9cdb-891b7ac6a857 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:11.094110 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8ef5b735-428e-4f85-9cdb-891b7ac6a857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:11.101073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1355/2733] 10.222.0.81 () {58 vars in 967 bytes} [Wed Aug 24 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:11.113740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:11.124907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4e436-b19c-4b51-b47c-b7543c6bd886 service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:11.130053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:11.130628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.130902 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.131173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.139911 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:11.141585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:11.149772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1379/2734] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:11.365453 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da736-1330-4540-8b35-5182fa086621 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.365735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da736-1330-4540-8b35-5182fa086621 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.366005 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da736-1330-4540-8b35-5182fa086621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.385431 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-308da736-1330-4540-8b35-5182fa086621 None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:11.386689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-308da736-1330-4540-8b35-5182fa086621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:11.393622 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1356/2735] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Aug 24 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:11.405407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:11.416072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3df3-8646-46d0-99b6-70e629e5a5ec service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:11.421083 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:11.421663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.421941 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.422214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.430266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:11.432463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:11.441667 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1380/2736] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:11.758842 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.759125 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.759396 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.778010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] MFA Rules not processed for user `cbd35279778f4858808385011249d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:11.779259 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:11.786183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1357/2737] 10.222.0.81 () {58 vars in 967 bytes} [Wed Aug 24 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:11.798812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:11.809933 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979428-4280-48ab-975b-6cef79bbf168 service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:11.814798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125804fc-5081-4fca-bf61-7662c4da645f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8eb54303a94ec38dbb9209c71381da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:11.815373 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125804fc-5081-4fca-bf61-7662c4da645f service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.815645 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125804fc-5081-4fca-bf61-7662c4da645f service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.815912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125804fc-5081-4fca-bf61-7662c4da645f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.823820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125804fc-5081-4fca-bf61-7662c4da645f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:11.825428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125804fc-5081-4fca-bf61-7662c4da645f service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:11.833593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1381/2738] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:13.907567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:13.909191 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:13.974705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06865-6161-48d1-a756-8faf8e74ee2c None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:13.979605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:13.980221 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:13.980501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:13.980823 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] PATH_INFO: `/v3/users/467f45f0cfdb4ef58f893e633075e3ae` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:13.981983 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=467f45f0cfdb4ef58f893e633075e3ae)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:13.983889 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.029841 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467f45f0cfdb4ef58f893e633075e3ae'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.037298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467f45f0cfdb4ef58f893e633075e3ae'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.037985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467f45f0cfdb4ef58f893e633075e3ae'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.045941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467f45f0cfdb4ef58f893e633075e3ae'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.058636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467f45f0cfdb4ef58f893e633075e3ae'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.060072 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:14.065625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1358/2739] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:13 2022] DELETE /identity/v3/users/467f45f0cfdb4ef58f893e633075e3ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:14.070097 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.070580 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.070694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.089425 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:14.090633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.150271 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1382/2740] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:14.160082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.161554 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.220602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09357f93-1854-4071-9bfd-96b214b93251 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.224393 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33d114-cff6-494c-8b85-14d0309829fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.225031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d114-cff6-494c-8b85-14d0309829fc service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.225327 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d114-cff6-494c-8b85-14d0309829fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.225586 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d114-cff6-494c-8b85-14d0309829fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.235518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33d114-cff6-494c-8b85-14d0309829fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.236963 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33d114-cff6-494c-8b85-14d0309829fc service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.246592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1359/2741] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:14.413818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.415258 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.428355 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221fbd7-bdd1-446a-8af5-340d5ab1ab27 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.433609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.434246 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.434527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.434855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] PATH_INFO: `/v3/projects/d5e2d5c2e0ce43f9b352daee2f2bf5f9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.436077 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e2d5c2e0ce43f9b352daee2f2bf5f9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.438114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.475907 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:14.488909 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e2d5c2e0ce43f9b352daee2f2bf5f9', 'request_id': 'req-9739b77c-da33-4535-b394-6fe70d2518b0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.495173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e2d5c2e0ce43f9b352daee2f2bf5f9', 'request_id': 'req-9739b77c-da33-4535-b394-6fe70d2518b0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.502006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e2d5c2e0ce43f9b352daee2f2bf5f9', 'request_id': 'req-9739b77c-da33-4535-b394-6fe70d2518b0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.512720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] The token cache is being invalidate because project d5e2d5c2e0ce43f9b352daee2f2bf5f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:14.513036 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.513668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9739b77c-da33-4535-b394-6fe70d2518b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.515153 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1383/2742] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:14 2022] DELETE /identity/v3/projects/d5e2d5c2e0ce43f9b352daee2f2bf5f9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:14.520047 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.520359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.520633 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.541410 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:14.542911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.601160 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1360/2743] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:14.608746 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.609055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.609333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.628656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:14.629947 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.638426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1384/2744] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:14.642511 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.652725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed92a7-f9a7-46f1-94d1-5063db5ae225 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.657258 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.657795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.658067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.658334 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.658971 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:14.659272 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.660876 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.672825 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1361/2745] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:14.677054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.686270 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f218f-12cf-4167-9c53-8fd89930d1c7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.690887 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.691470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.691747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.692008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.692624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.694553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.724018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1385/2746] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:14.728149 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.738686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac142a60-94a9-4011-bc3e-3aa52191a278 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.796012 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.796615 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.796924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.797195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.797816 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.799749 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.822608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1362/2747] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:14.826733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.836946 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6fb20-87c7-4d35-9c08-5edf838dda15 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.842538 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.843123 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.843395 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.843662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.844289 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.845914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.855678 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1386/2748] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:14.859548 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.869292 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71793-b2f4-4a63-a25b-f51141994655 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.873732 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331af1b6-e6ac-4e43-8499-dab4556322a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.874304 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331af1b6-e6ac-4e43-8499-dab4556322a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.874577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331af1b6-e6ac-4e43-8499-dab4556322a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.874840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331af1b6-e6ac-4e43-8499-dab4556322a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.875469 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331af1b6-e6ac-4e43-8499-dab4556322a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.877071 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331af1b6-e6ac-4e43-8499-dab4556322a5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.887695 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1363/2749] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:14.891658 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.902507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08a9b3-859e-435e-ab90-97bba5bedc48 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.907417 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.908006 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.908284 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.908559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] PATH_INFO: `/v3/projects/69f494a609f241328d85790280b204b8/users/5143313cddbb4f56bf48d84167069c74/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.923656 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f494a609f241328d85790280b204b8, user_id=5143313cddbb4f56bf48d84167069c74, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.926421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.939976 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1387/2750] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:14 2022] PUT /identity/v3/projects/69f494a609f241328d85790280b204b8/users/5143313cddbb4f56bf48d84167069c74/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:14.945122 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61a810-8b34-4127-96bf-0d36ddaabf2d None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.945598 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61a810-8b34-4127-96bf-0d36ddaabf2d None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.945719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61a810-8b34-4127-96bf-0d36ddaabf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.959722 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:14.966627 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-ff61a810-8b34-4127-96bf-0d36ddaabf2d None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:14.967832 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ff61a810-8b34-4127-96bf-0d36ddaabf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:15.025940 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1364/2751] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:15.035965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:15.037743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:15.095056 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e926d-f955-43f5-bb1c-13f00d710fdf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:15.099002 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:15.099664 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.099939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.100209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:15.109313 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:15.110747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:15.120598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1388/2752] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:15.719793 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56692af9-5567-49ea-85bf-3f04982349d2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.720081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56692af9-5567-49ea-85bf-3f04982349d2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.720406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56692af9-5567-49ea-85bf-3f04982349d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:15.749251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-56692af9-5567-49ea-85bf-3f04982349d2 None None] MFA Rules not processed for user `5143313cddbb4f56bf48d84167069c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:15.750544 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-56692af9-5567-49ea-85bf-3f04982349d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:15.818354 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1365/2753] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:15.829294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:15.830706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:15.891740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6838b2f-3062-4127-bc9d-09f20aa69d90 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:15.895730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:15.896332 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.896609 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.896916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:15.907634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:15.909108 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:15.919416 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1389/2754] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:17.274797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:17.276081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:17.276638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:17.296246 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] MFA Rules not processed for user `5143313cddbb4f56bf48d84167069c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:17.297809 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:17.304372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1366/2755] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:25.914042 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:27.289673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=59 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:50.766760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:50.776435 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95e4d-1981-4140-8261-ff83e4c1cec6 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:50.780020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:50.780706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:50.781017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:50.781287 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:50.783024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:50.795638 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:50.797186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:50.807315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1390/2756] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:54.068152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.069783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:54.091564 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ecbc16-5976-45ab-8ff7-c31a6661314a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.096924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.098017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.098514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.098943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] PATH_INFO: `/v3/users/1af20de2433c44ff8fb1f16290404c21` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.100426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af20de2433c44ff8fb1f16290404c21)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.102655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.159325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af20de2433c44ff8fb1f16290404c21'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.169459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af20de2433c44ff8fb1f16290404c21'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.170528 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af20de2433c44ff8fb1f16290404c21'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.180111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af20de2433c44ff8fb1f16290404c21'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.196242 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-1d0858c6-7c0d-4619-bbfa-f91318646d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af20de2433c44ff8fb1f16290404c21'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.203903 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1367/2757] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:54 2022] DELETE /identity/v3/users/1af20de2433c44ff8fb1f16290404c21 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:54.404863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.418082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e934-f16d-4736-a594-40ad1b465114 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.479136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.479872 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.480150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.480413 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] PATH_INFO: `/v3/projects/b5da9e76bed44f3d90144e4fbff2cb1c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.481573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5da9e76bed44f3d90144e4fbff2cb1c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.483481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.529431 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5da9e76bed44f3d90144e4fbff2cb1c', 'request_id': 'req-d57d3883-ad8b-4031-bf18-75bb27ce25d0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.535400 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5da9e76bed44f3d90144e4fbff2cb1c', 'request_id': 'req-d57d3883-ad8b-4031-bf18-75bb27ce25d0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.543075 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5da9e76bed44f3d90144e4fbff2cb1c', 'request_id': 'req-d57d3883-ad8b-4031-bf18-75bb27ce25d0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.553338 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] The token cache is being invalidate because project b5da9e76bed44f3d90144e4fbff2cb1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:54.553622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.554241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d57d3883-ad8b-4031-bf18-75bb27ce25d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:54.555693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1391/2758] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:54 2022] DELETE /identity/v3/projects/b5da9e76bed44f3d90144e4fbff2cb1c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:54.561602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.561894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.562175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.587313 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:54.589056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:54.653455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1368/2759] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:54.661661 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.661943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.662214 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.683964 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:54.685208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:54.693729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1392/2760] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:54.698068 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.708206 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a115038-a293-4a79-bb8f-f31c21ddb3d3 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.713249 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.714156 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.714460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.714734 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.715374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:54.715672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.717943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.730992 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1369/2761] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:54.735327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.753024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc13309-220c-41fa-90d8-21ab0f9255fa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.757860 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.758449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.758726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.758997 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.759791 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.762268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.792107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1393/2762] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:54 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:54.796264 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.807066 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be706f-ddff-490f-80c0-d1c4ed3e4256 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.863863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.864455 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.864754 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.865058 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.865706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.867664 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.875910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:54.889778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1370/2763] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:54 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:54.894016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.905437 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2bf37-511c-4766-9973-a5b72414854e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.910184 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.910838 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.911119 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.911386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.912019 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.913653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.924705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1394/2764] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:54.928956 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.940688 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9ddbe-370d-47e6-af94-3c7ef40bb214 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.945624 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad621b5-b368-4dc8-a215-81bd638afde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.946349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad621b5-b368-4dc8-a215-81bd638afde5 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.946634 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad621b5-b368-4dc8-a215-81bd638afde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.946907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad621b5-b368-4dc8-a215-81bd638afde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.947758 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad621b5-b368-4dc8-a215-81bd638afde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:54.949408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad621b5-b368-4dc8-a215-81bd638afde5 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:54.960580 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1371/2765] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:54.965404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:54.975758 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a0291-a740-4a00-a62a-7cdab82f7666 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:54.981925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:54.982591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:54.982868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:54.983143 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] PATH_INFO: `/v3/projects/4b05b1de2f064ba29a6d0f7f4c8b2a01/users/e9fa457d74c644a3ad740da44576a12f/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:54.998290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b05b1de2f064ba29a6d0f7f4c8b2a01, user_id=e9fa457d74c644a3ad740da44576a12f, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:55.001007 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:55.013993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1395/2766] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:54 2022] PUT /identity/v3/projects/4b05b1de2f064ba29a6d0f7f4c8b2a01/users/e9fa457d74c644a3ad740da44576a12f/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:55.019029 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361435c7-2638-41dc-a5c3-3ab4c14e94ae None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:55.019309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361435c7-2638-41dc-a5c3-3ab4c14e94ae None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:55.019599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361435c7-2638-41dc-a5c3-3ab4c14e94ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:55.038460 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-361435c7-2638-41dc-a5c3-3ab4c14e94ae None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:55.039647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-361435c7-2638-41dc-a5c3-3ab4c14e94ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:55.098740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1372/2767] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:55.108969 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:55.110649 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:55.125191 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:55.171634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f782c5-d79c-4b23-b487-59f69cd24d10 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:55.175329 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:55.175922 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:55.176205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:55.176475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:55.185550 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:55.187009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:55.196559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1396/2768] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:59.171707 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f63bb-226a-4d60-9b9a-66cfa751afab None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:59.172204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f63bb-226a-4d60-9b9a-66cfa751afab None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:59.172305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f63bb-226a-4d60-9b9a-66cfa751afab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:59.205783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-cf3f63bb-226a-4d60-9b9a-66cfa751afab None None] MFA Rules not processed for user `e9fa457d74c644a3ad740da44576a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:59.207370 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-cf3f63bb-226a-4d60-9b9a-66cfa751afab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:59.284280 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1373/2769] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:37:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:59.301573 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:59.307928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:59.408954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f383d3-8ff9-4520-b5f5-5f17c4ea9dbf service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:59.414729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550fe32-3f37-422c-9439-9a2827cc67b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:59.415339 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550fe32-3f37-422c-9439-9a2827cc67b2 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:59.415623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550fe32-3f37-422c-9439-9a2827cc67b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:59.415967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550fe32-3f37-422c-9439-9a2827cc67b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:59.425122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550fe32-3f37-422c-9439-9a2827cc67b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:59.426767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550fe32-3f37-422c-9439-9a2827cc67b2 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:59.437251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1397/2770] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:00.825289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:00.825783 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:00.825881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:00.845111 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] MFA Rules not processed for user `e9fa457d74c644a3ad740da44576a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:00.846312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:00.853128 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1374/2771] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:38:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:09.432572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=54 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:10.839567 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:20.950921 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0fcb-9f09-406a-9ab0-a0ec12624838 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.951438 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0fcb-9f09-406a-9ab0-a0ec12624838 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.951546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0fcb-9f09-406a-9ab0-a0ec12624838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.981592 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d87a0fcb-9f09-406a-9ab0-a0ec12624838 None None] MFA Rules not processed for user `5143313cddbb4f56bf48d84167069c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:20.982808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d87a0fcb-9f09-406a-9ab0-a0ec12624838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:21.057085 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1398/2772] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:38:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:31.054982 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=37 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:38.958924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:38.960374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.098699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad8f8e-855a-4c27-b0d9-dce4d382aa17 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:39.104224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:39.104837 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.105142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.105407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.108056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.119391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:39.120994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:39.130795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1375/2773] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:49.126665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=63 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:02.021365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:02.021912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:02.021912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:02.040939 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] MFA Rules not processed for user `e9fa457d74c644a3ad740da44576a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:02.042144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:02.050596 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1399/2774] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:12.036031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:17.562458 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:17.564630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:17.614785 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c688d-7d41-4aba-82fb-fcf5e519e548 service placement] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:17.622002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:17.622652 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:17.622927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:17.623338 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] PATH_INFO: `/v3/users/5143313cddbb4f56bf48d84167069c74` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:17.624530 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5143313cddbb4f56bf48d84167069c74)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:17.627107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:17.679765 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5143313cddbb4f56bf48d84167069c74'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:17.686879 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5143313cddbb4f56bf48d84167069c74'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:17.687570 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5143313cddbb4f56bf48d84167069c74'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:17.695670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5143313cddbb4f56bf48d84167069c74'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:17.708481 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-d0d72d40-d3ec-4a21-8571-997dd8b0ca59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5143313cddbb4f56bf48d84167069c74'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:17.714592 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1376/2775] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:39:17 2022] DELETE /identity/v3/users/5143313cddbb4f56bf48d84167069c74 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:17.894952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:17.907439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b613fc-7024-49f8-838d-e0bedf1fa01b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:17.964912 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:17.965469 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:17.965747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:17.966022 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] PATH_INFO: `/v3/projects/69f494a609f241328d85790280b204b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:17.967136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f494a609f241328d85790280b204b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:17.970023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:18.015871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f494a609f241328d85790280b204b8', 'request_id': 'req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:18.021896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f494a609f241328d85790280b204b8', 'request_id': 'req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:18.029358 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f494a609f241328d85790280b204b8', 'request_id': 'req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:18.041383 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] The token cache is being invalidate because project 69f494a609f241328d85790280b204b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:39:18.041707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:18.042394 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:18.043955 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1400/2776] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:39:17 2022] DELETE /identity/v3/projects/69f494a609f241328d85790280b204b8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:27.711321 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:28.038507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:04.199546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:04.200055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:04.200163 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:04.219583 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] MFA Rules not processed for user `e9fa457d74c644a3ad740da44576a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:04.221108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:04.276152 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1377/2777] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:40:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:11.812205 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:11.813761 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:11.894477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803cf4a-5ea0-47fc-8d2e-33c313bf72be admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:11.898137 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:11.898741 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:11.899020 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:11.899294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:11.901067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:11.989722 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:11.991150 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:12.010812 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1401/2778] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:13.195289 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:13.204988 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ae69c-d6ff-4f1f-9546-742ecc456998 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:13.208602 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:13.209231 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:13.209512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:13.209784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:13.211574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:13.226297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:13.227719 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:13.239408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1378/2779] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:22.008277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=75 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:23.233803 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=31 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:28.072551 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.074132 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.142750 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.144906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.171720 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1573a-5c71-46ed-bd1c-8949a329650a service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.177916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.178535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.178814 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.179090 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.180871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.194067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.194968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.206568 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1402/2780] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:28.233740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0d713-a1d0-4671-8245-c2558f8f0c19 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.240678 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.241273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.241557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.241836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] PATH_INFO: `/v3/users/e5588121cc364d30a8f01d17c4cfc6df` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.243061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5588121cc364d30a8f01d17c4cfc6df)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.245174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.292324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5588121cc364d30a8f01d17c4cfc6df'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.300009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5588121cc364d30a8f01d17c4cfc6df'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.300672 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5588121cc364d30a8f01d17c4cfc6df'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.307850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5588121cc364d30a8f01d17c4cfc6df'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.319863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5588121cc364d30a8f01d17c4cfc6df'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.325694 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1379/2781] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:28 2022] DELETE /identity/v3/users/e5588121cc364d30a8f01d17c4cfc6df => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:40:28.335973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.337436 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.350152 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712246-67d9-46ec-a1af-e53627cdf303 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.398633 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.399221 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.399494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.399767 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.401458 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.466099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.467475 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.478148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1403/2782] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:28.651419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.653008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.666512 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cb191-1fe0-4f4c-b064-e81e608f066e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.671131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.671774 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.672055 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.672329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] PATH_INFO: `/v3/projects/5dea7fb32dea4f75917d82973bf7e7e6` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.673475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dea7fb32dea4f75917d82973bf7e7e6)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.676062 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.726550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dea7fb32dea4f75917d82973bf7e7e6', 'request_id': 'req-ccd44dee-fab6-4d59-adb9-102465b92841'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.732665 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dea7fb32dea4f75917d82973bf7e7e6', 'request_id': 'req-ccd44dee-fab6-4d59-adb9-102465b92841'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.740846 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dea7fb32dea4f75917d82973bf7e7e6', 'request_id': 'req-ccd44dee-fab6-4d59-adb9-102465b92841'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.751339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] The token cache is being invalidate because project 5dea7fb32dea4f75917d82973bf7e7e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:28.751632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.752330 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.753905 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1380/2783] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:28 2022] DELETE /identity/v3/projects/5dea7fb32dea4f75917d82973bf7e7e6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:28.757896 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.767553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729ccca-bcbc-4949-a118-5dbde42bf464 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.806806 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:28.825067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.825684 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.825958 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.826228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] PATH_INFO: `/v3/users/eba4b2752d54437ea93823e52f76b32b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.827325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba4b2752d54437ea93823e52f76b32b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.829283 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.874900 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba4b2752d54437ea93823e52f76b32b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.882254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba4b2752d54437ea93823e52f76b32b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.895934 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba4b2752d54437ea93823e52f76b32b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.896614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba4b2752d54437ea93823e52f76b32b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.903637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba4b2752d54437ea93823e52f76b32b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.912779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1404/2784] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:28 2022] DELETE /identity/v3/users/eba4b2752d54437ea93823e52f76b32b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:29.084056 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.085623 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:29.093725 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:29.101337 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd44dee-fab6-4d59-adb9-102465b92841 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.162170 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.162869 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.163061 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.163333 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] PATH_INFO: `/v3/projects/9d0d9fcf69f04ad99a8c1e68eea71735` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.164514 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0d9fcf69f04ad99a8c1e68eea71735)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.166451 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.214011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0d9fcf69f04ad99a8c1e68eea71735', 'request_id': 'req-289cee6b-4031-477a-b7ca-6eaca741441e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.220190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0d9fcf69f04ad99a8c1e68eea71735', 'request_id': 'req-289cee6b-4031-477a-b7ca-6eaca741441e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.228195 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0d9fcf69f04ad99a8c1e68eea71735', 'request_id': 'req-289cee6b-4031-477a-b7ca-6eaca741441e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.239566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] The token cache is being invalidate because project 9d0d9fcf69f04ad99a8c1e68eea71735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:29.239878 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.240508 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-289cee6b-4031-477a-b7ca-6eaca741441e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.241999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1381/2785] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:29 2022] DELETE /identity/v3/projects/9d0d9fcf69f04ad99a8c1e68eea71735 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:38.908363 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:39.235764 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:00.365653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:00.367490 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:00.465680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb522f-acd3-46e0-8df7-a088d865f3ca admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:00.553669 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:00.554290 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:00.554561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:00.554828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:00.557773 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:00.571357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:00.574280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:00.586024 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1405/2786] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:03.260608 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:03.261121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:03.261248 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:03.280954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] MFA Rules not processed for user `e9fa457d74c644a3ad740da44576a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:03.282169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:03.338784 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1382/2787] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:41:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:03.975765 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:03.978105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:04.063832 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8afb99-b0a5-4b40-9bf6-6f13f0abeb8a service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:04.069041 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:04.069668 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:04.070078 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:04.070353 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:04.072430 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:04.083935 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:04.085554 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:04.096402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1406/2788] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:05.902965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:05.904518 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:05.967397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9c20f-b997-4ab1-8bde-f89e35a0bc34 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:05.971205 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:05.971799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:05.972073 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:05.972339 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:05.974085 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:05.984478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:05.985912 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:05.994506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1383/2789] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:14.092050 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:15.990342 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=51 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:06.813450 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:06.824316 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5eafb-b3b0-44fe-ab19-f39554bd3abc service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:06.829575 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:06.830322 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:06.830695 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:06.831048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:06.832925 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:06.845622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:06.847465 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:06.862614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1407/2790] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:06.866295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:06.868401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:06.970278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6253a1-b18b-4456-85df-746b6ce4cfe9 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:06.975894 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:06.976513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:06.976808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:06.977082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] PATH_INFO: `/v3/users/e9fa457d74c644a3ad740da44576a12f` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:06.978210 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fa457d74c644a3ad740da44576a12f)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:06.980121 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.031201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fa457d74c644a3ad740da44576a12f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.038397 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fa457d74c644a3ad740da44576a12f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.039048 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fa457d74c644a3ad740da44576a12f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.046301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fa457d74c644a3ad740da44576a12f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.058314 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-49497631-bbaf-41e3-aa7b-37e1bd00a739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fa457d74c644a3ad740da44576a12f'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.064086 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1384/2791] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:06 2022] DELETE /identity/v3/users/e9fa457d74c644a3ad740da44576a12f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:42:07.232875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.244318 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138dba-b031-4b3a-af57-ff950740060d service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.304548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.305120 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.305393 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.305660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] PATH_INFO: `/v3/projects/4b05b1de2f064ba29a6d0f7f4c8b2a01` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.306760 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b05b1de2f064ba29a6d0f7f4c8b2a01)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.308694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.356315 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b05b1de2f064ba29a6d0f7f4c8b2a01', 'request_id': 'req-def66307-1247-4e45-b5b3-f79d32b01a17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.362574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b05b1de2f064ba29a6d0f7f4c8b2a01', 'request_id': 'req-def66307-1247-4e45-b5b3-f79d32b01a17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.370392 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b05b1de2f064ba29a6d0f7f4c8b2a01', 'request_id': 'req-def66307-1247-4e45-b5b3-f79d32b01a17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.381170 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] The token cache is being invalidate because project 4b05b1de2f064ba29a6d0f7f4c8b2a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:07.381463 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.382157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-def66307-1247-4e45-b5b3-f79d32b01a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:07.383710 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1408/2792] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:07 2022] DELETE /identity/v3/projects/4b05b1de2f064ba29a6d0f7f4c8b2a01 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:07.388778 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.389088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.389379 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.449659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:07.450957 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:07.495804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:07.513051 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1385/2793] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:07.521480 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.521871 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.522165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.541798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:07.542998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:07.550785 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1409/2794] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:07.555146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.564787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef157f-3e9e-47a7-a78f-a6a782a8270e None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.569850 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.570445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.570723 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.570994 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.571637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:42:07.571936 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.573578 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.585710 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1386/2795] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:42:07.590370 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.599891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69c31-775a-40f6-8e2b-0eda00143847 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.604585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.605203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.605501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.605770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.606378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.608256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.645268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1410/2796] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:07 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:07.649173 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.659008 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839cba-c4b5-4560-9109-6ece4054531d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.713614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.714198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.714473 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.714739 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.715612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.717606 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.738704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1387/2797] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:07 2022] POST /identity/v3/users => generated 279 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:07.743335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.753647 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741981fe-d74a-43da-8fdf-0010b3e2ad0e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.759340 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.759926 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.760203 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.760500 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.761169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.762764 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.774018 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1411/2798] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:07.778115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.789605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6dca7-65cc-4b6b-8af5-ec23fd41becc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.794491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8382c-a5ca-43b0-b90c-827d89a68146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.795082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8382c-a5ca-43b0-b90c-827d89a68146 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.795359 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8382c-a5ca-43b0-b90c-827d89a68146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.795632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8382c-a5ca-43b0-b90c-827d89a68146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.796271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8382c-a5ca-43b0-b90c-827d89a68146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.797934 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8382c-a5ca-43b0-b90c-827d89a68146 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.809944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1388/2799] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:07.814069 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.825883 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645c675-a21c-4d6f-9f68-11ffe0b13d0b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:07.830563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:07.831224 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.831501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.831774 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] PATH_INFO: `/v3/projects/6cdf454001584f6f94f69d37d46c86c6/users/c357810835a644b9a103ef72f0386784/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.847726 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cdf454001584f6f94f69d37d46c86c6, user_id=c357810835a644b9a103ef72f0386784, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:07.851169 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:07.863790 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1412/2800] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:07 2022] PUT /identity/v3/projects/6cdf454001584f6f94f69d37d46c86c6/users/c357810835a644b9a103ef72f0386784/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:07.874053 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:07.874536 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:07.874638 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:07.902412 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] MFA Rules not processed for user `c357810835a644b9a103ef72f0386784`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:07.903657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:07.966815 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1389/2801] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:07.971277 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:07.977660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:08.036386 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1cae8-86d4-45b9-a587-3e82d5eedc50 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.042565 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.043162 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.043439 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.043709 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.044401 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.046378 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.072324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1413/2802] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:07 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:08.076372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.088187 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317a9ad-6558-4302-8f1c-91206d3002d6 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.129978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:08.146953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.147851 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.148130 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.148399 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.149614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.150962 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.173501 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1390/2803] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:08 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:42:08.179675 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.190165 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d820ca0-5284-4e6e-965b-d46331767d99 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.195910 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.196563 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.196863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.197155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.197817 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.199782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.213335 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1414/2804] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:08.217756 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.228150 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fbafb-4545-4253-97c0-e03194b8c85d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.233201 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.233852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.234133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.234428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] PATH_INFO: `/v3/projects/7ca7a35d15c044a3b4f3a70f04e4ef42/users/ff3895c33b284e9487d899bcb3d919c7/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.250830 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca7a35d15c044a3b4f3a70f04e4ef42, user_id=ff3895c33b284e9487d899bcb3d919c7, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.253489 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.266693 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1391/2805] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:08 2022] PUT /identity/v3/projects/7ca7a35d15c044a3b4f3a70f04e4ef42/users/ff3895c33b284e9487d899bcb3d919c7/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:08.270780 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.281932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7bb20-6c8d-41df-8888-71fbc9f709ec admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.337823 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb89c84-dc2e-492f-8358-81f00701b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.338333 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb89c84-dc2e-492f-8358-81f00701b0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.338618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb89c84-dc2e-492f-8358-81f00701b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.338892 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb89c84-dc2e-492f-8358-81f00701b0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.339527 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb89c84-dc2e-492f-8358-81f00701b0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.341432 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb89c84-dc2e-492f-8358-81f00701b0f9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.351327 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1415/2806] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:08.355574 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.365910 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c57c5-c09b-49d5-add8-4adafc475ec2 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.370491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.371126 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.371407 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.371682 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] PATH_INFO: `/v3/projects/7ca7a35d15c044a3b4f3a70f04e4ef42/users/ff3895c33b284e9487d899bcb3d919c7/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.373626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca7a35d15c044a3b4f3a70f04e4ef42, user_id=ff3895c33b284e9487d899bcb3d919c7, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.376263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.387302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1392/2807] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:08 2022] PUT /identity/v3/projects/7ca7a35d15c044a3b4f3a70f04e4ef42/users/ff3895c33b284e9487d899bcb3d919c7/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:08.399727 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.400008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.400282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.428189 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] MFA Rules not processed for user `ff3895c33b284e9487d899bcb3d919c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:08.429501 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:08.492886 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:08.497106 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1416/2808] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:08.508653 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.510419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:08.591390 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544104c2-b159-427f-ab56-647bb9a7469e admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.595083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.595673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.595952 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.596224 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.605356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.606895 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.616668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1393/2809] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:08.691472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.704875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1ddd-a867-4e86-9218-6cb734ae0fc6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.708375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f7559-13e8-4525-bdd2-87fee7237402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.709054 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f7559-13e8-4525-bdd2-87fee7237402 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.709597 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f7559-13e8-4525-bdd2-87fee7237402 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.710005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f7559-13e8-4525-bdd2-87fee7237402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.767995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f7559-13e8-4525-bdd2-87fee7237402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.769474 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f7559-13e8-4525-bdd2-87fee7237402 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.778426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1417/2810] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:18.612038 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=59,keystone:INSERT=5 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:18.774379 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=28 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:23.546637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:23.609220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c835c39-fc9b-4720-a6c2-2486b242581d service cinder] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:23.613595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:23.614204 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:23.614478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:23.614744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] PATH_INFO: `/v3/users/c357810835a644b9a103ef72f0386784` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:23.616513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c357810835a644b9a103ef72f0386784)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:23.618441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:23.666275 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c357810835a644b9a103ef72f0386784'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:23.673115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c357810835a644b9a103ef72f0386784'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:23.673772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c357810835a644b9a103ef72f0386784'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:23.681704 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c357810835a644b9a103ef72f0386784'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:23.693576 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c357810835a644b9a103ef72f0386784'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:23.700500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1394/2811] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:23 2022] DELETE /identity/v3/users/c357810835a644b9a103ef72f0386784 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:23.704985 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:23.705455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:23.705562 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:23.724890 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:23.726613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:23.785828 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1418/2812] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:23.796033 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:23.797438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:23.854577 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af268-9535-4ba9-9b83-55e0b4a87aa8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:23.858236 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:23.858834 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:23.859115 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:23.859384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:23.869142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:23.870546 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:23.879965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1395/2813] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:24.148088 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:24.149916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:24.164427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0802-88b4-4eb2-b439-a31bfd577f1a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:24.169839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:24.170530 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:24.170772 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:24.171046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] PATH_INFO: `/v3/projects/6cdf454001584f6f94f69d37d46c86c6` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:24.172452 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cdf454001584f6f94f69d37d46c86c6)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:24.174903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:24.221483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cdf454001584f6f94f69d37d46c86c6', 'request_id': 'req-93310d5a-e736-4749-b8d4-b92b0f3172d8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.228023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cdf454001584f6f94f69d37d46c86c6', 'request_id': 'req-93310d5a-e736-4749-b8d4-b92b0f3172d8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.235406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cdf454001584f6f94f69d37d46c86c6', 'request_id': 'req-93310d5a-e736-4749-b8d4-b92b0f3172d8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.246319 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] The token cache is being invalidate because project 6cdf454001584f6f94f69d37d46c86c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:24.246598 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.247238 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.248798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1419/2814] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:24 2022] DELETE /identity/v3/projects/6cdf454001584f6f94f69d37d46c86c6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:24.252826 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:24.263155 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa654f-b761-410e-9bc6-07a869569074 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:24.281738 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:24.323174 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:24.323721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:24.323999 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:24.324273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] PATH_INFO: `/v3/users/ff3895c33b284e9487d899bcb3d919c7` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:24.325436 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3895c33b284e9487d899bcb3d919c7)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:24.327358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:24.375954 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3895c33b284e9487d899bcb3d919c7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.383617 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3895c33b284e9487d899bcb3d919c7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.384298 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3895c33b284e9487d899bcb3d919c7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.391852 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3895c33b284e9487d899bcb3d919c7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.404104 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-f0cd7545-ca8c-4ddb-99da-76cc0a918b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3895c33b284e9487d899bcb3d919c7'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.409987 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1396/2815] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:24 2022] DELETE /identity/v3/users/ff3895c33b284e9487d899bcb3d919c7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:24.681779 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:24.683209 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:24.697262 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310d5a-e736-4749-b8d4-b92b0f3172d8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:24.756655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:24.757381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:24.757601 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:24.757873 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] PATH_INFO: `/v3/projects/7ca7a35d15c044a3b4f3a70f04e4ef42` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:24.759034 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca7a35d15c044a3b4f3a70f04e4ef42)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:24.760989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:24.791357 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:24.808680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca7a35d15c044a3b4f3a70f04e4ef42', 'request_id': 'req-d2e0c12f-930b-4fca-bbfc-fe408e563112'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.814808 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca7a35d15c044a3b4f3a70f04e4ef42', 'request_id': 'req-d2e0c12f-930b-4fca-bbfc-fe408e563112'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.821967 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca7a35d15c044a3b4f3a70f04e4ef42', 'request_id': 'req-d2e0c12f-930b-4fca-bbfc-fe408e563112'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.832254 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] The token cache is being invalidate because project 7ca7a35d15c044a3b4f3a70f04e4ef42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:24.832536 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.833166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2e0c12f-930b-4fca-bbfc-fe408e563112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:24.834682 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1420/2816] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:24 2022] DELETE /identity/v3/projects/7ca7a35d15c044a3b4f3a70f04e4ef42 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:24.839892 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:24.840350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:24.840445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:24.859711 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:24.860944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:24.921340 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1397/2817] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:24.932122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:24.932406 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:24.932713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:24.952548 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:24.953801 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:24.961804 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1421/2818] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:24.966388 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:24.977211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874bbf8-8fdd-47b3-8b8d-ec4c50db5a10 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:24.981675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:24.982263 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:24.982539 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:24.982811 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] PATH_INFO: `/v3/domains` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:24.983449 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81486) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:42:24.983747 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:24.985394 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:24.997955 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1398/2819] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:42:25.002188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.012067 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e83b39-7ae4-4489-8531-5a5b284b1def None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.016670 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.017288 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.017567 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.017839 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.018549 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.020457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.048855 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1422/2820] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:25 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:25.052939 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.062795 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4638b9a-564b-451f-80c0-a15c7b6466de admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.089772 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:25.122513 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.123039 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.123320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.123595 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.124225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.126211 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.146566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1399/2821] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:25 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:25.151747 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.162109 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ca35-1dd3-4d0e-aff8-efbd4b5f8798 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.167512 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.168099 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.168372 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.168640 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.169308 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.170916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.181113 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1423/2822] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:25.185040 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.195060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3969244-7900-4664-bcdb-bfb6f72b4786 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.199625 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34c8c6-6280-4b1f-855b-dff543856bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.200220 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34c8c6-6280-4b1f-855b-dff543856bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.200500 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34c8c6-6280-4b1f-855b-dff543856bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.200806 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34c8c6-6280-4b1f-855b-dff543856bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.201442 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34c8c6-6280-4b1f-855b-dff543856bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.203103 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34c8c6-6280-4b1f-855b-dff543856bfa admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.215802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1400/2823] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:25.221297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.232346 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4d36-951c-4433-adad-342b88bfabe1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.236977 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.237618 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.237893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.238166 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] PATH_INFO: `/v3/projects/1238195c03f04a3abb3d10d718ad6040/users/a2057eda1c364b9f8de6b1144242faa7/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.253938 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1238195c03f04a3abb3d10d718ad6040, user_id=a2057eda1c364b9f8de6b1144242faa7, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.256663 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.269745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1424/2824] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:25 2022] PUT /identity/v3/projects/1238195c03f04a3abb3d10d718ad6040/users/a2057eda1c364b9f8de6b1144242faa7/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:25.332717 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.333028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.333302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.361550 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] MFA Rules not processed for user `a2057eda1c364b9f8de6b1144242faa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:25.362762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:25.426196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1401/2825] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:25.430698 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.493693 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfc7c-9322-4587-a3cd-4e244891584c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.498325 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.498923 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.499196 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.499464 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.500072 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.502017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.528782 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1425/2826] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:25 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:25.532858 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.543613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04628d2-5f01-49a7-ac4f-58b4d36b9f3a None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.604516 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.605212 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.605491 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.605762 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] PATH_INFO: `/v3/users` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.606384 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.608301 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.630293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1402/2827] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:25 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:25.634975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.645347 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb59b7-7b1b-4940-8a03-051afb2b9e59 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.650766 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.651360 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.651637 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.651952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.652561 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.654199 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.665525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1426/2828] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:25.669668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.680701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd8d19-648e-4226-a457-1a5e705c8af8 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.685255 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.685888 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.686166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.686464 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] PATH_INFO: `/v3/projects/fb92f4bd9efa49a190ff6976fd0e799e/users/d02fe191d9f74a83a985a61a28547f95/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.703965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb92f4bd9efa49a190ff6976fd0e799e, user_id=d02fe191d9f74a83a985a61a28547f95, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.706701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.720380 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1403/2829] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:25 2022] PUT /identity/v3/projects/fb92f4bd9efa49a190ff6976fd0e799e/users/d02fe191d9f74a83a985a61a28547f95/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:25.724279 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.734440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1f9ae-dfd1-4a45-a2aa-3ba7427f8f65 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.754299 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:25.792110 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dae7b-2950-4e25-a89f-2dde8b11e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.792798 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dae7b-2950-4e25-a89f-2dde8b11e39b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.792990 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dae7b-2950-4e25-a89f-2dde8b11e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.793259 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dae7b-2950-4e25-a89f-2dde8b11e39b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.793884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dae7b-2950-4e25-a89f-2dde8b11e39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.795473 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dae7b-2950-4e25-a89f-2dde8b11e39b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.805622 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1427/2830] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:25.809881 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.815165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:25.822278 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dfd10-be26-4d1c-94f5-4e3a061131bc admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:25.826741 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:25.827372 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.827643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.827911 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] PATH_INFO: `/v3/projects/fb92f4bd9efa49a190ff6976fd0e799e/users/d02fe191d9f74a83a985a61a28547f95/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.829813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb92f4bd9efa49a190ff6976fd0e799e, user_id=d02fe191d9f74a83a985a61a28547f95, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:25.832426 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:25.842735 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1404/2831] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:25 2022] PUT /identity/v3/projects/fb92f4bd9efa49a190ff6976fd0e799e/users/d02fe191d9f74a83a985a61a28547f95/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:25.852621 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:25.852942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:25.853226 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:25.882263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] MFA Rules not processed for user `d02fe191d9f74a83a985a61a28547f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:25.883952 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:25.957870 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1428/2832] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:25.968535 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:25.970267 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:26.029169 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bff020-eb57-44e1-9da8-ea3f9c123997 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:26.032829 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476a5fa-afef-46c5-9148-3783d3602c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:26.033427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476a5fa-afef-46c5-9148-3783d3602c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:26.033708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476a5fa-afef-46c5-9148-3783d3602c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:26.033981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476a5fa-afef-46c5-9148-3783d3602c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:26.042831 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476a5fa-afef-46c5-9148-3783d3602c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:26.044251 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476a5fa-afef-46c5-9148-3783d3602c8d service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:26.054233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1405/2833] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:26.459936 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:26.523010 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2cf56-7dc8-457f-b946-ecf6135ddbeb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:26.527730 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:26.528345 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:26.528653 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:26.528962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] PATH_INFO: `/v3/users/a2057eda1c364b9f8de6b1144242faa7` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:26.530084 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2057eda1c364b9f8de6b1144242faa7)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:26.532012 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:26.577756 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2057eda1c364b9f8de6b1144242faa7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:26.579433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:26.586402 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2057eda1c364b9f8de6b1144242faa7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:26.598796 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2057eda1c364b9f8de6b1144242faa7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:26.599433 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2057eda1c364b9f8de6b1144242faa7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:26.606974 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2057eda1c364b9f8de6b1144242faa7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:26.612752 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1429/2834] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:26 2022] DELETE /identity/v3/users/a2057eda1c364b9f8de6b1144242faa7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:26.617515 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:26.617913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:26.618284 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:26.637052 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:26.638237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:26.697787 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1406/2835] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:26.707494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:26.708965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:26.767585 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a00469-9f43-44c5-90e1-52b41df05518 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:26.771148 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:26.771743 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:26.772023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:26.772297 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:26.781245 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:26.782694 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:26.792479 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1430/2836] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:27.069435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:27.070958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:27.085297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a0fa8-dfb0-4878-8eeb-78231f33e193 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:27.090133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:27.091296 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.091585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.092017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] PATH_INFO: `/v3/projects/1238195c03f04a3abb3d10d718ad6040` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.093193 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1238195c03f04a3abb3d10d718ad6040)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:27.095107 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:27.141820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1238195c03f04a3abb3d10d718ad6040', 'request_id': 'req-eb3388ff-ad6c-4243-98db-8967ec9b7555'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.149059 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1238195c03f04a3abb3d10d718ad6040', 'request_id': 'req-eb3388ff-ad6c-4243-98db-8967ec9b7555'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.156609 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1238195c03f04a3abb3d10d718ad6040', 'request_id': 'req-eb3388ff-ad6c-4243-98db-8967ec9b7555'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.167024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] The token cache is being invalidate because project 1238195c03f04a3abb3d10d718ad6040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:27.167316 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.167943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.169457 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1407/2837] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:27 2022] DELETE /identity/v3/projects/1238195c03f04a3abb3d10d718ad6040 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:27.173424 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:27.183107 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f0ae6-5139-4cb4-9201-8fd5e49b144c service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:27.241300 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:27.241928 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.242208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.242481 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] PATH_INFO: `/v3/users/d02fe191d9f74a83a985a61a28547f95` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.243614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02fe191d9f74a83a985a61a28547f95)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:27.245600 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:27.297291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02fe191d9f74a83a985a61a28547f95'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.305236 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02fe191d9f74a83a985a61a28547f95'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.316235 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:27.317460 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02fe191d9f74a83a985a61a28547f95'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.318096 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02fe191d9f74a83a985a61a28547f95'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.327499 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ecb50aac-dd2b-4ff4-ab43-42990488b78d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02fe191d9f74a83a985a61a28547f95'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.333384 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1431/2838] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:27 2022] DELETE /identity/v3/users/d02fe191d9f74a83a985a61a28547f95 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:27.605007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:27.606445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:27.620176 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3388ff-ad6c-4243-98db-8967ec9b7555 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:27.633194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:27.683142 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:27.683769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.684043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.684312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] PATH_INFO: `/v3/projects/fb92f4bd9efa49a190ff6976fd0e799e` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.685446 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb92f4bd9efa49a190ff6976fd0e799e)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:27.687371 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:27.733953 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb92f4bd9efa49a190ff6976fd0e799e', 'request_id': 'req-72042325-f448-4bb7-b2a5-83b6dc886a8e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.740064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb92f4bd9efa49a190ff6976fd0e799e', 'request_id': 'req-72042325-f448-4bb7-b2a5-83b6dc886a8e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.748655 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb92f4bd9efa49a190ff6976fd0e799e', 'request_id': 'req-72042325-f448-4bb7-b2a5-83b6dc886a8e'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.759699 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] The token cache is being invalidate because project fb92f4bd9efa49a190ff6976fd0e799e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:27.759984 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.760612 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-72042325-f448-4bb7-b2a5-83b6dc886a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:27.762112 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1408/2839] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:27 2022] DELETE /identity/v3/projects/fb92f4bd9efa49a190ff6976fd0e799e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:27.766973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.767263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.767574 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.789217 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:27.790472 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:27.851825 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1432/2840] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:27.859919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.860203 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.860478 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.880736 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:27.882102 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:27.889814 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1409/2841] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:27.894009 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:27.903770 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb3b7d-5d5c-4aa2-a53d-701f2cc2b4bc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:27.908252 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:27.908863 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.909127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.909399 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.910035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:42:27.910331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:27.911916 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:27.924317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1433/2842] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:42:27.930007 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:27.939217 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b5deb-bba8-4e8c-b430-9e30aee78b9b None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:27.943915 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:27.944495 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:27.944791 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:27.945075 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:27.945708 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:27.947636 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:27.974009 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1410/2843] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:27 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:42:27.978810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:27.988623 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84174c93-df84-44fc-a51f-9cb0d8506efb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:28.046210 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:28.046811 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:28.047095 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:28.047365 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:28.047989 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:28.049954 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:28.071128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1434/2844] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:27 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:28.075190 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:28.086227 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7084b72-08eb-44da-8748-d79d30e62fd9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:28.091092 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:28.091680 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:28.091959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:28.092233 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:28.092909 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:28.094504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:28.104978 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1411/2845] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:28.109102 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:28.119613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb62fa-8730-4d02-8208-261248ad4ee4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:28.124591 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013bb4d-5679-4e42-b434-325e1be214ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:28.125206 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013bb4d-5679-4e42-b434-325e1be214ee admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:28.125487 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013bb4d-5679-4e42-b434-325e1be214ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:28.125763 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013bb4d-5679-4e42-b434-325e1be214ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:28.126397 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013bb4d-5679-4e42-b434-325e1be214ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:28.128001 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013bb4d-5679-4e42-b434-325e1be214ee admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:28.138867 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1435/2846] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:28.142797 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:28.153341 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139d2b-019e-4c7c-b31c-237100121dc9 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:28.158438 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:28.159083 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:28.159365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:28.159644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] PATH_INFO: `/v3/projects/bcdb09d8369d469aae3c17ee322dcc80/users/602f97330a764874ac5190cd5446fa9e/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:28.175046 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdb09d8369d469aae3c17ee322dcc80, user_id=602f97330a764874ac5190cd5446fa9e, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:28.177794 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:28.190534 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1412/2847] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:28 2022] PUT /identity/v3/projects/bcdb09d8369d469aae3c17ee322dcc80/users/602f97330a764874ac5190cd5446fa9e/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:28.202208 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:28.202532 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:28.202793 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:28.231028 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] MFA Rules not processed for user `602f97330a764874ac5190cd5446fa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:28.232241 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:28.274320 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:28.298201 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1436/2848] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:28.309786 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:28.311454 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:28.360558 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:28.375312 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e865d-462e-487f-aa7b-e93fba32a7d0 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:28.378965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156b6a7-7dcb-460f-86c0-2fdb42278b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:28.379553 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156b6a7-7dcb-460f-86c0-2fdb42278b2e service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:28.379827 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156b6a7-7dcb-460f-86c0-2fdb42278b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:28.380091 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156b6a7-7dcb-460f-86c0-2fdb42278b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:28.388868 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156b6a7-7dcb-460f-86c0-2fdb42278b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:28.390281 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156b6a7-7dcb-460f-86c0-2fdb42278b2e service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:28.399349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1413/2849] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:42:35.948705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:36.012455 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14d8bd-e2c3-40a9-8aa6-9ecfe0d83291 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:36.016995 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:36.017611 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.017893 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.018164 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] PATH_INFO: `/v3/users/602f97330a764874ac5190cd5446fa9e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.019298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=602f97330a764874ac5190cd5446fa9e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:36.021265 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:36.070576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602f97330a764874ac5190cd5446fa9e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.078213 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602f97330a764874ac5190cd5446fa9e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.091144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602f97330a764874ac5190cd5446fa9e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.091789 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602f97330a764874ac5190cd5446fa9e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.098751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602f97330a764874ac5190cd5446fa9e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.105112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1437/2850] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:35 2022] DELETE /identity/v3/users/602f97330a764874ac5190cd5446fa9e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:36.109698 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.110162 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.110305 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.130165 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:36.131365 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:36.189820 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1414/2851] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:36.200116 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:36.201607 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:36.259660 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c635-da03-42ea-adfc-ce440f9c6287 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:36.263317 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeee3f-44fe-4202-8bdf-bc255d75ad38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:36.263898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeee3f-44fe-4202-8bdf-bc255d75ad38 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.264180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeee3f-44fe-4202-8bdf-bc255d75ad38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.264477 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeee3f-44fe-4202-8bdf-bc255d75ad38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.273608 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeee3f-44fe-4202-8bdf-bc255d75ad38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:36.275046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeee3f-44fe-4202-8bdf-bc255d75ad38 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:36.284546 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1438/2852] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:36.571262 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:36.572965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:36.586060 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950440fa-55f9-46d0-af3c-2247470aa6d2 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:36.591443 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:36.592074 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.592348 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.592613 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] PATH_INFO: `/v3/projects/bcdb09d8369d469aae3c17ee322dcc80` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.593770 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdb09d8369d469aae3c17ee322dcc80)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:36.595673 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:36.644375 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdb09d8369d469aae3c17ee322dcc80', 'request_id': 'req-7ab40ebd-1ba4-4fba-9576-cc455058271c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.651175 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdb09d8369d469aae3c17ee322dcc80', 'request_id': 'req-7ab40ebd-1ba4-4fba-9576-cc455058271c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.658525 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdb09d8369d469aae3c17ee322dcc80', 'request_id': 'req-7ab40ebd-1ba4-4fba-9576-cc455058271c'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.669401 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] The token cache is being invalidate because project bcdb09d8369d469aae3c17ee322dcc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:36.669706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.670374 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-7ab40ebd-1ba4-4fba-9576-cc455058271c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:36.671872 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1415/2853] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:36 2022] DELETE /identity/v3/projects/bcdb09d8369d469aae3c17ee322dcc80 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:36.676799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.677086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.677381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.686194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:36.699799 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:36.701055 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:36.766849 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1439/2854] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:36.775616 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.775907 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.776186 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.798918 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:36.800194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:36.809320 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1416/2855] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:36.813948 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:36.824222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d83d-0c07-4c6c-9acf-8dfcf7aabd1f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:36.829060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:36.829685 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.829962 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.830233 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.830876 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:42:36.831179 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:36.832898 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:36.845146 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1440/2856] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:42:36.849475 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:36.859427 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183b923-8be4-40bf-a20e-c78b3108e0ef None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:36.864133 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:36.864721 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.865031 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.865302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.865959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:36.868647 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:36.895435 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1417/2857] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:36 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:36.899738 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:36.910440 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632545ec-c2c2-4c6f-8985-cc29349f089e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:36.968445 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:36.969105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:36.969266 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:36.969537 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:36.970168 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:36.972105 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:36.992924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1441/2858] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:36 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:36.997317 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:37.007662 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99673a-0f5f-4108-80d7-2fdddee2a52c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:37.012788 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:37.013378 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:37.013657 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:37.013923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:37.014556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:37.016532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:37.027081 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1418/2859] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:37.031065 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:37.041877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f33daf-7c49-4c7b-901c-4058794afe7c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:37.046508 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c588-4b96-476a-a34b-aa201baae03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:37.047736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c588-4b96-476a-a34b-aa201baae03c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:37.048021 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c588-4b96-476a-a34b-aa201baae03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:37.048298 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c588-4b96-476a-a34b-aa201baae03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:37.048973 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8c588-4b96-476a-a34b-aa201baae03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:37.050572 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8c588-4b96-476a-a34b-aa201baae03c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:37.061230 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1442/2860] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:37.065406 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:37.075556 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea18a6e-c21e-4d91-a50f-76b16828c76b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:37.080148 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:37.080810 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:37.081095 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:37.081369 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] PATH_INFO: `/v3/projects/ac7a67edb87747819cf06069dda23e90/users/3a8e4f05e2884b75b9036517908995b8/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:37.100139 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7a67edb87747819cf06069dda23e90, user_id=3a8e4f05e2884b75b9036517908995b8, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:37.103676 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:37.117346 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1419/2861] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:37 2022] PUT /identity/v3/projects/ac7a67edb87747819cf06069dda23e90/users/3a8e4f05e2884b75b9036517908995b8/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:37.133543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:37.133820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:37.134094 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:37.171251 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] MFA Rules not processed for user `3a8e4f05e2884b75b9036517908995b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:37.172755 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:37.238412 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:37.242818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1443/2862] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:37.253692 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:37.255504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:37.262409 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:37.313459 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1eb503-ecef-4945-8895-1e6c2cbfed0d admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:37.317070 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2384a-1085-4ee9-92c6-108dc602218e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06578427361b421fa275befc898a7f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:37.317656 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2384a-1085-4ee9-92c6-108dc602218e service cinder] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:37.317931 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2384a-1085-4ee9-92c6-108dc602218e service cinder] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:37.318200 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2384a-1085-4ee9-92c6-108dc602218e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:37.326714 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2384a-1085-4ee9-92c6-108dc602218e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:37.328131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2384a-1085-4ee9-92c6-108dc602218e service cinder] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:37.337423 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1420/2863] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:47.332875 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=24 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:53.566506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:53.629787 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ed7ea-0823-4aeb-bcb0-6d6a1b586c60 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:53.634834 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:53.635467 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:53.635751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:53.636023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] PATH_INFO: `/v3/users/3a8e4f05e2884b75b9036517908995b8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:53.637194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8e4f05e2884b75b9036517908995b8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:53.639173 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:53.685843 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8e4f05e2884b75b9036517908995b8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:53.693032 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8e4f05e2884b75b9036517908995b8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:53.705387 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8e4f05e2884b75b9036517908995b8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:53.706046 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8e4f05e2884b75b9036517908995b8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:53.713802 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8e4f05e2884b75b9036517908995b8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:53.719706 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1444/2864] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:42:53 2022] DELETE /identity/v3/users/3a8e4f05e2884b75b9036517908995b8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:53.724315 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:53.724799 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:53.726101 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:53.744916 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:53.746114 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:53.805445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1421/2865] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:53.814965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:53.816451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:53.875136 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502a23c-9828-4d36-8324-3a0244c65ba0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:53.879023 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea34439-08de-45a7-8aa3-8dc1745102b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:53.879657 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea34439-08de-45a7-8aa3-8dc1745102b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:53.879950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea34439-08de-45a7-8aa3-8dc1745102b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:53.880222 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea34439-08de-45a7-8aa3-8dc1745102b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:53.890875 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea34439-08de-45a7-8aa3-8dc1745102b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:53.892326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea34439-08de-45a7-8aa3-8dc1745102b1 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:53.903493 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1445/2866] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:54.173884 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.175293 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:54.187990 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10fc96-6ece-4304-88c6-e7c622ea0953 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.192614 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.193331 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.193604 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.193906 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] PATH_INFO: `/v3/projects/ac7a67edb87747819cf06069dda23e90` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.195024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7a67edb87747819cf06069dda23e90)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.196959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.244349 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7a67edb87747819cf06069dda23e90', 'request_id': 'req-4d22d317-8a5a-4bd4-a848-4b85cab104bc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:54.250675 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7a67edb87747819cf06069dda23e90', 'request_id': 'req-4d22d317-8a5a-4bd4-a848-4b85cab104bc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:54.257659 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7a67edb87747819cf06069dda23e90', 'request_id': 'req-4d22d317-8a5a-4bd4-a848-4b85cab104bc'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:54.268049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] The token cache is being invalidate because project ac7a67edb87747819cf06069dda23e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:42:54.268358 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:54.269028 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-4d22d317-8a5a-4bd4-a848-4b85cab104bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:42:54.270529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1422/2867] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:42:54 2022] DELETE /identity/v3/projects/ac7a67edb87747819cf06069dda23e90 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:54.276005 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.276282 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.276553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.296533 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:54.297768 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:54.314729 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:54.359035 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1446/2868] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:54.366924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.367208 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.367482 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.389637 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:54.390877 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:54.399282 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1423/2869] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:54.403564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.414578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c8b2d-f893-4a46-95da-9036a8c32b46 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.419263 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.420011 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.420312 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.420581 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.421256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:42:54.421553 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.423183 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.435323 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1447/2870] 10.222.0.81 () {58 vars in 1153 bytes} [Wed Aug 24 03:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:42:54.439792 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.449207 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f3f4d-9852-48f4-a366-9025d1f050d9 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.453802 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.454391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.454670 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.454941 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.455590 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.457533 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.486127 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1424/2871] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:54 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:54.490326 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.500114 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a462a-7074-4194-b066-671b19af889d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.555879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.556470 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.556748 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.557053 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.557662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.559620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.579810 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1448/2872] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:54 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:54.583861 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.595098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f0566-9100-494b-90e7-54c8f3a0f156 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.599658 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.600919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.601198 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.601470 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.602108 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.603703 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.613432 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1425/2873] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:54.617381 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.627535 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e620-6fda-49d6-a1a9-57f764dd585b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.633013 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b9832-23c5-4154-b904-49a2f90d4296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.633628 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b9832-23c5-4154-b904-49a2f90d4296 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.633914 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b9832-23c5-4154-b904-49a2f90d4296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.634188 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b9832-23c5-4154-b904-49a2f90d4296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.634818 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b9832-23c5-4154-b904-49a2f90d4296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.636486 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b9832-23c5-4154-b904-49a2f90d4296 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.647850 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1449/2874] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:54.651949 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.661828 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5aa8-0088-478c-84c1-c7bdaf96858c admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.666645 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.667283 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.667566 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.667843 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] PATH_INFO: `/v3/projects/c742ff2fa942409384b6294cf883ce0a/users/d32bf7e9512f4a51ae52a25bba18b891/roles/fed56dd461254bc682cf1955076ea721` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.683146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c742ff2fa942409384b6294cf883ce0a, user_id=d32bf7e9512f4a51ae52a25bba18b891, role_id=fed56dd461254bc682cf1955076ea721)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.685849 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.699135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1426/2875] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:54 2022] PUT /identity/v3/projects/c742ff2fa942409384b6294cf883ce0a/users/d32bf7e9512f4a51ae52a25bba18b891/roles/fed56dd461254bc682cf1955076ea721 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:42:54.704134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd5e58-9a5d-4f03-8620-e9e00aceb1be None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.704427 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd5e58-9a5d-4f03-8620-e9e00aceb1be None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.704733 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd5e58-9a5d-4f03-8620-e9e00aceb1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.724249 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-26cd5e58-9a5d-4f03-8620-e9e00aceb1be None None] MFA Rules not processed for user `5a8025e93c084f56aeda2462957536b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:54.726080 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-26cd5e58-9a5d-4f03-8620-e9e00aceb1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:54.784174 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1450/2876] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:54.794197 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:54.795998 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:54.817644 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:54.854930 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928877e-28d4-46a5-aee3-93368689c779 admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:54.858504 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:54.859096 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:54.859368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:54.859635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:54.868463 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:54.869924 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:54.879131 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1427/2877] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:58.901129 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:58.901625 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:58.901736 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:58.933449 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] MFA Rules not processed for user `d32bf7e9512f4a51ae52a25bba18b891`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:58.934975 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:58.954943 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:59.010593 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1451/2878] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:42:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:59.015166 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:59.025808 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34b0f1-66c9-4418-83b8-c0d4051b6f85 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:59.030450 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:59.031043 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:59.031322 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:59.031594 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:59.032245 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:59.034167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:59.064687 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1428/2879] 10.222.0.81 () {60 vars in 1153 bytes} [Wed Aug 24 03:42:59 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:42:59.068484 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:59.080348 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcd9a0-1d0d-4203-8b1b-07c2991b453c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:59.146256 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:59.147421 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:59.147713 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:59.147979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:59.148603 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:59.150571 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:59.170930 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1452/2880] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Aug 24 03:42:59 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:42:59.175013 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:59.186239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4ae45-72ba-4f2e-ae4a-becb8d358bdb admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:59.190773 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:59.191355 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:59.191632 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:59.191900 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:59.192532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:59.194686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:59.205099 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1429/2881] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:59.210404 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:59.220711 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5309239-51d6-475d-8d7b-a2c3eed5410e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:59.225788 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:59.226426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:59.226701 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:59.226968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] PATH_INFO: `/v3/projects/5bc0639a6318486bb0aa5ab9e8cc9ced/users/9aa935d3b9254660abca3d299dfcc2fb/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:59.242239 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc0639a6318486bb0aa5ab9e8cc9ced, user_id=9aa935d3b9254660abca3d299dfcc2fb, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:59.244965 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:59.259031 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1453/2882] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:59 2022] PUT /identity/v3/projects/5bc0639a6318486bb0aa5ab9e8cc9ced/users/9aa935d3b9254660abca3d299dfcc2fb/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:42:59.263004 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:59.273874 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36887a61-476f-4dac-bbd7-50effd9fb7ce admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:59.333408 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023afd29-b31a-4a4f-80fe-fdc712e193ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:59.334011 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023afd29-b31a-4a4f-80fe-fdc712e193ab admin admin] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:59.335347 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023afd29-b31a-4a4f-80fe-fdc712e193ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:59.335630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023afd29-b31a-4a4f-80fe-fdc712e193ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:59.336271 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023afd29-b31a-4a4f-80fe-fdc712e193ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:59.337904 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023afd29-b31a-4a4f-80fe-fdc712e193ab admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:59.347643 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1430/2883] 10.222.0.81 () {58 vars in 1122 bytes} [Wed Aug 24 03:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:42:59.352539 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:59.365177 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6c965-eb20-4abd-a6b7-df1519501e3b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:59.371375 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:59.372093 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:59.372367 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:59.372634 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] PATH_INFO: `/v3/projects/5bc0639a6318486bb0aa5ab9e8cc9ced/users/9aa935d3b9254660abca3d299dfcc2fb/roles/bf40971954c544fd896cd08b9e6e5230` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:59.374564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc0639a6318486bb0aa5ab9e8cc9ced, user_id=9aa935d3b9254660abca3d299dfcc2fb, role_id=bf40971954c544fd896cd08b9e6e5230)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:59.377269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:59.388144 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1454/2884] 10.222.0.81 () {60 vars in 1483 bytes} [Wed Aug 24 03:42:59 2022] PUT /identity/v3/projects/5bc0639a6318486bb0aa5ab9e8cc9ced/users/9aa935d3b9254660abca3d299dfcc2fb/roles/bf40971954c544fd896cd08b9e6e5230 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:43:03.757020 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d024c-b015-48cc-9236-83e4b62970eb None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:03.757526 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d024c-b015-48cc-9236-83e4b62970eb None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:03.757642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d024c-b015-48cc-9236-83e4b62970eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:03.802088 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-574d024c-b015-48cc-9236-83e4b62970eb None None] MFA Rules not processed for user `9aa935d3b9254660abca3d299dfcc2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:03.803840 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-574d024c-b015-48cc-9236-83e4b62970eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:03.917921 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1431/2885] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:43:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:03.940868 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:03.942841 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:04.034624 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cefb6e-a71a-445b-9b7e-22ca39935ed6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:04.040642 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:04.041280 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:04.041555 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:04.041816 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:04.047740 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:04.115451 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:04.117047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:04.127073 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1455/2886] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:13.913563 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:14.122426 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=24 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:20.460557 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:20.461082 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:20.461183 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:20.480605 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] MFA Rules not processed for user `d32bf7e9512f4a51ae52a25bba18b891`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:20.481870 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:20.488701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1432/2887] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:43:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:20.803180 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:20.804620 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:20.861324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa044a6-149e-4776-8984-4cf3e4382882 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:20.864924 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:20.866127 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:20.866428 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:20.866702 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:20.868407 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:20.877692 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:20.879122 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:20.887356 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1456/2888] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:30.475445 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=3 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:30.883666 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=26 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:34.510194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:34.511838 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:34.531800 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8db324-0f64-4147-b2af-92a87db1fae8 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:34.538439 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:34.539630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:34.539630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:34.539630 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:34.541635 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:34.557302 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:34.560356 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:34.579767 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1433/2889] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:36.788264 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:36.797390 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fbc48-4e46-472a-99d5-12bb19a0bfee service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:36.800983 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:36.801564 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:36.801877 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:36.802154 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:36.811160 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:36.812614 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:36.822576 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1457/2890] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:38.629167 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:38.641763 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0db255-66d9-4169-853f-10ba9b479a68 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:38.647728 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:38.648350 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:38.648628 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:38.648945 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:38.650807 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:38.664403 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:38.666064 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:38.678421 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1434/2891] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:46.817826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:48.672666 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=12 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:23.312835 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:23.314228 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:23.384610 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7dd7f-986c-4a77-a557-324fa04b298a service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:23.390268 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:23.390865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:23.391142 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:23.391415 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:23.393255 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:23.404994 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:23.406577 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:23.416453 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1458/2892] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:33.412187 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=32 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:32.938744 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:32.948804 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b1f6-85b0-40e1-8500-442edfdbac42 service nova] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:32.952392 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:32.953017 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:32.953297 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:32.953568 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:32.955325 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:32.970520 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:32.971959 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167090ef-7c04-4577-a523-75a08b9d497e service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:32.985777 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1435/2893] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:34.171505 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.181157 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b28ab-7e17-42b8-b778-13c5a57ee7b8 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.184942 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.185526 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.185795 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.186060 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.187751 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:34.200884 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.202304 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.214368 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1459/2894] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:36.497796 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] REQUEST_METHOD: `POST` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:36.498309 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:36.498413 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:36.518689 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.auth.core [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] MFA Rules not processed for user `d32bf7e9512f4a51ae52a25bba18b891`. Rule list: `[]` (Enabled: `True`). {{(pid=81486) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:36.520237 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:36.527981 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1436/2895] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:45:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:36.841566 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:36.850134 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e99b8-69e7-41f3-89da-88c101a22890 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:36.853707 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:36.854503 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:36.854778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:36.855047 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:36.856725 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:36.866025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:36.867496 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15fc2a-e49f-4b1e-828b-3ceea122f881 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:36.875528 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1460/2896] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:46.512985 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=9 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:46.871968 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=12 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:32.007345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:32.018010 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ca20e-fcd5-414c-a833-f570fec4c5a1 None None] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:32.021441 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:32.022024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:32.022295 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:32.022560 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:32.024306 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:32.034168 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:32.035599 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:32.043529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1437/2897] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:42.039960 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:47:28.293291 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:47:28.293826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:47:28.293826 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:47:28.314112 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] MFA Rules not processed for user `d32bf7e9512f4a51ae52a25bba18b891`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:47:28.315324 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:28.322139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1461/2898] 10.222.0.81 () {58 vars in 958 bytes} [Wed Aug 24 03:47:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:47:29.088626 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:47:29.090194 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:29.107324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0003ad-3415-41a8-8014-68b2ffca42b6 service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:47:29.114146 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:47:29.114813 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:47:29.115225 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:47:29.115506 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:47:29.117345 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:29.129686 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:47:29.131328 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:47:29.142067 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1438/2899] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:47:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:47:31.469082 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:47:31.484038 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aba0b0-6e3c-4e61-8ecc-30e83161c084 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:47:31.489903 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8afb31c267946b7851cb1c22b62b6cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:47:31.490494 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:47:31.490778 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:47:31.491048 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:47:31.492984 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:31.507853 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:47:31.509331 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:47:31.522721 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1462/2900] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:47:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:47:39.137135 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=6 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:47:41.518543 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=9 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:24.866006 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:24.884958 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e977cb2-823f-4e20-8d49-ed84f3990ff5 service placement] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:24.889057 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:24.889642 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:24.889913 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:24.890179 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:24.913230 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:24.914661 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:24.923923 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1439/2901] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:28.319361 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.431809 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46ba93-2fa4-4f3a-a4af-aee5eee46c0f service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.438998 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.439613 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.439891 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.440163 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.453559 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.455194 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.466681 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1463/2902] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:34.919391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=17 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:38.009381 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:38.019093 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa5897-2f5c-430d-b68f-c39bca00568c service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:38.022742 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a2363821f445db980712d3e491e791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:38.023428 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:38.023740 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:38.024018 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:38.097585 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:38.099022 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:38.118180 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1440/2903] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:38.462016 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=27 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:41.214970 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:41.229821 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13422ce-1d2e-4b60-bcfa-020ea8e137d8 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:41.235950 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:41.236560 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:41.236999 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:41.237186 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:41.239079 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:41.288956 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:41.292076 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:41.304745 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1464/2904] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:44.372706 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:44.390419 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c950076-7ceb-4125-8fde-cb1a1a7f41ee service neutron] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:44.395919 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:44.396529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] REQUEST_METHOD: `GET` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:44.396835 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:44.397353 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:44.399106 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:44.418440 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:44.420589 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:44.433943 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1441/2905] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:44.472590 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:44.483155 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d468b-0e0e-4042-80ff-b4de659bdb0e service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:44.495008 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:44.495655 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:44.495929 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:44.496198 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] PATH_INFO: `/v3/users/d32bf7e9512f4a51ae52a25bba18b891` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:44.497403 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32bf7e9512f4a51ae52a25bba18b891)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:44.499865 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:44.562128 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32bf7e9512f4a51ae52a25bba18b891'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.574662 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32bf7e9512f4a51ae52a25bba18b891'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.594680 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32bf7e9512f4a51ae52a25bba18b891'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.595507 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32bf7e9512f4a51ae52a25bba18b891'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.603269 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32bf7e9512f4a51ae52a25bba18b891'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.609086 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1465/2906] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:44 2022] DELETE /identity/v3/users/d32bf7e9512f4a51ae52a25bba18b891 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:44.778639 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:44.794529 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cbd83-8eba-4220-96ee-40d5cc229c15 service placement] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:44.858063 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:44.858685 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:44.858965 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:44.859239 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] PATH_INFO: `/v3/projects/c742ff2fa942409384b6294cf883ce0a` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:44.860367 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c742ff2fa942409384b6294cf883ce0a)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:44.862329 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:44.912540 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c742ff2fa942409384b6294cf883ce0a', 'request_id': 'req-0e487d6e-610a-49a4-841b-a7607a42b50b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.918769 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c742ff2fa942409384b6294cf883ce0a', 'request_id': 'req-0e487d6e-610a-49a4-841b-a7607a42b50b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.927002 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c742ff2fa942409384b6294cf883ce0a', 'request_id': 'req-0e487d6e-610a-49a4-841b-a7607a42b50b'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.930836 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:44.953243 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] The token cache is being invalidate because project c742ff2fa942409384b6294cf883ce0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:44.953532 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.954196 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:44.955668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1442/2907] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:44 2022] DELETE /identity/v3/projects/c742ff2fa942409384b6294cf883ce0a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:44.959691 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:44.961139 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:44.974294 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6518a2c-44b5-45af-842c-52ff53374f69 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:45.030769 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:45.031306 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:45.031578 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:45.031851 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] PATH_INFO: `/v3/users/9aa935d3b9254660abca3d299dfcc2fb` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:45.033025 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa935d3b9254660abca3d299dfcc2fb)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:45.034963 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:45.076248 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:45.082879 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa935d3b9254660abca3d299dfcc2fb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.090979 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa935d3b9254660abca3d299dfcc2fb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.102836 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa935d3b9254660abca3d299dfcc2fb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.103483 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa935d3b9254660abca3d299dfcc2fb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.110993 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa935d3b9254660abca3d299dfcc2fb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.116705 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1466/2908] 10.222.0.81 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:44 2022] DELETE /identity/v3/users/9aa935d3b9254660abca3d299dfcc2fb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:45.304324 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Authenticating user token {{(pid=81486) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:45.305893 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.fernet_utils [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81486) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:45.324273 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e487d6e-610a-49a4-841b-a7607a42b50b admin admin] Validating token access rules against request {{(pid=81486) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:45.381705 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a8025e93c084f56aeda2462957536b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c28eab9a294b9ba17b7f5b9c24d868', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81486) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:45.382368 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:45.382588 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:45.382863 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] PATH_INFO: `/v3/projects/5bc0639a6318486bb0aa5ab9e8cc9ced` {{(pid=81486) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:45.383982 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc0639a6318486bb0aa5ab9e8cc9ced)` {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:45.385927 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] RBAC: Authorization granted {{(pid=81486) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:45.433098 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc0639a6318486bb0aa5ab9e8cc9ced', 'request_id': 'req-68172e68-5ddf-4fb1-9de0-065f54dec0cf'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.439701 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc0639a6318486bb0aa5ab9e8cc9ced', 'request_id': 'req-68172e68-5ddf-4fb1-9de0-065f54dec0cf'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.446668 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc0639a6318486bb0aa5ab9e8cc9ced', 'request_id': 'req-68172e68-5ddf-4fb1-9de0-065f54dec0cf'} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.458024 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] The token cache is being invalidate because project 5bc0639a6318486bb0aa5ab9e8cc9ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81486) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:45.458310 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.458944 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG keystone.notifications [None req-68172e68-5ddf-4fb1-9de0-065f54dec0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81486) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:45.460391 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: [pid: 81486|app: 0|req: 1443/2909] 10.222.0.81 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:45 2022] DELETE /identity/v3/projects/5bc0639a6318486bb0aa5ab9e8cc9ced => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:55.113376 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:55.455049 dv-cinder-854261-1-iscsi devstack@keystone.service[81486]: DEBUG dbcounter [-] [81486] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=81486) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:35.257525 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:35.258978 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:35.335523 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15389c52-6ac0-4748-8050-077516f19755 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:35.406506 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bae100a3a8e94ca6b2800df52ab453f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7c18d17b92a4f839c7f6b179443c794', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:35.407197 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:35.407457 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:35.407744 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:35.410017 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:35.422257 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:35.424820 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f13795-f416-40b7-889c-f143de321e24 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:35.436509 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1467/2910] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:45.430932 dv-cinder-854261-1-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=58 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}